draft-ietf-trans-gossip-02.txt | draft-ietf-trans-gossip-03.txt | |||
---|---|---|---|---|
TRANS L. Nordberg | TRANS L. Nordberg | |||
Internet-Draft NORDUnet | Internet-Draft NORDUnet | |||
Intended status: Experimental D. Gillmor | Intended status: Experimental D. Gillmor | |||
Expires: September 22, 2016 ACLU | Expires: January 9, 2017 ACLU | |||
T. Ritter | T. Ritter | |||
March 21, 2016 | July 08, 2016 | |||
Gossiping in CT | Gossiping in CT | |||
draft-ietf-trans-gossip-02 | draft-ietf-trans-gossip-03 | |||
Abstract | Abstract | |||
The logs in Certificate Transparency are untrusted in the sense that | The logs in Certificate Transparency are untrusted in the sense that | |||
the users of the system don't have to trust that they behave | the users of the system don't have to trust that they behave | |||
correctly since the behaviour of a log can be verified to be correct. | correctly since the behavior of a log can be verified to be correct. | |||
This document tries to solve the problem with logs presenting a | This document tries to solve the problem with logs presenting a | |||
"split view" of their operations. It describes three gossiping | "split view" of their operations. It describes three gossiping | |||
mechanisms for Certificate Transparency: SCT Feedback, STH | mechanisms for Certificate Transparency: SCT Feedback, STH | |||
Pollination and Trusted Auditor Relationship. | Pollination and Trusted Auditor Relationship. | |||
Status of This Memo | Status of This Memo | |||
This Internet-Draft is submitted in full conformance with the | This Internet-Draft is submitted in full conformance with the | |||
provisions of BCP 78 and BCP 79. | provisions of BCP 78 and BCP 79. | |||
skipping to change at page 1, line 40 ¶ | skipping to change at page 1, line 40 ¶ | |||
Internet-Drafts are working documents of the Internet Engineering | Internet-Drafts are working documents of the Internet Engineering | |||
Task Force (IETF). Note that other groups may also distribute | Task Force (IETF). Note that other groups may also distribute | |||
working documents as Internet-Drafts. The list of current Internet- | working documents as Internet-Drafts. The list of current Internet- | |||
Drafts is at http://datatracker.ietf.org/drafts/current/. | Drafts is at http://datatracker.ietf.org/drafts/current/. | |||
Internet-Drafts are draft documents valid for a maximum of six months | Internet-Drafts are draft documents valid for a maximum of six months | |||
and may be updated, replaced, or obsoleted by other documents at any | and may be updated, replaced, or obsoleted by other documents at any | |||
time. It is inappropriate to use Internet-Drafts as reference | time. It is inappropriate to use Internet-Drafts as reference | |||
material or to cite them other than as "work in progress." | material or to cite them other than as "work in progress." | |||
This Internet-Draft will expire on September 22, 2016. | This Internet-Draft will expire on January 9, 2017. | |||
Copyright Notice | Copyright Notice | |||
Copyright (c) 2016 IETF Trust and the persons identified as the | Copyright (c) 2016 IETF Trust and the persons identified as the | |||
document authors. All rights reserved. | document authors. All rights reserved. | |||
This document is subject to BCP 78 and the IETF Trust's Legal | This document is subject to BCP 78 and the IETF Trust's Legal | |||
Provisions Relating to IETF Documents | Provisions Relating to IETF Documents | |||
(http://trustee.ietf.org/license-info) in effect on the date of | (http://trustee.ietf.org/license-info) in effect on the date of | |||
publication of this document. Please review these documents | publication of this document. Please review these documents | |||
skipping to change at page 2, line 42 ¶ | skipping to change at page 2, line 42 ¶ | |||
8.3. Trusted Auditor Stream . . . . . . . . . . . . . . . . . 17 | 8.3. Trusted Auditor Stream . . . . . . . . . . . . . . . . . 17 | |||
8.3.1. Trusted Auditor data format . . . . . . . . . . . . . 18 | 8.3.1. Trusted Auditor data format . . . . . . . . . . . . . 18 | |||
9. 3-Method Ecosystem . . . . . . . . . . . . . . . . . . . . . 19 | 9. 3-Method Ecosystem . . . . . . . . . . . . . . . . . . . . . 19 | |||
9.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 19 | 9.1. SCT Feedback . . . . . . . . . . . . . . . . . . . . . . 19 | |||
9.2. STH Pollination . . . . . . . . . . . . . . . . . . . . . 20 | 9.2. STH Pollination . . . . . . . . . . . . . . . . . . . . . 20 | |||
9.3. Trusted Auditor Relationship . . . . . . . . . . . . . . 21 | 9.3. Trusted Auditor Relationship . . . . . . . . . . . . . . 21 | |||
9.4. Interaction . . . . . . . . . . . . . . . . . . . . . . . 22 | 9.4. Interaction . . . . . . . . . . . . . . . . . . . . . . . 22 | |||
10. Security considerations . . . . . . . . . . . . . . . . . . . 22 | 10. Security considerations . . . . . . . . . . . . . . . . . . . 22 | |||
10.1. Attacks by actively malicious logs . . . . . . . . . . . 22 | 10.1. Attacks by actively malicious logs . . . . . . . . . . . 22 | |||
10.2. Dual-CA Compromise . . . . . . . . . . . . . . . . . . . 23 | 10.2. Dual-CA Compromise . . . . . . . . . . . . . . . . . . . 23 | |||
10.3. Censorship/Blocking considerations . . . . . . . . . . . 23 | 10.3. Censorship/Blocking considerations . . . . . . . . . . . 24 | |||
10.4. Privacy considerations . . . . . . . . . . . . . . . . . 25 | 10.4. Flushing Attacks . . . . . . . . . . . . . . . . . . . . 25 | |||
10.4.1. Privacy and SCTs . . . . . . . . . . . . . . . . . . 25 | 10.4.1. STHs . . . . . . . . . . . . . . . . . . . . . . . . 25 | |||
10.4.2. Privacy in SCT Feedback . . . . . . . . . . . . . . 25 | 10.4.2. SCTs & Certificate Chains on HTTPS Servers . . . . . 26 | |||
10.4.3. Privacy for HTTPS clients performing STH Proof | 10.4.3. SCTs & Certificate Chains on HTTPS Clients . . . . . 26 | |||
Fetching . . . . . . . . . . . . . . . . . . . . . . 26 | 10.5. Privacy considerations . . . . . . . . . . . . . . . . . 27 | |||
10.4.4. Privacy in STH Pollination . . . . . . . . . . . . . 26 | 10.5.1. Privacy and SCTs . . . . . . . . . . . . . . . . . . 27 | |||
10.4.5. Privacy in STH Interaction . . . . . . . . . . . . . 27 | 10.5.2. Privacy in SCT Feedback . . . . . . . . . . . . . . 27 | |||
10.4.6. Trusted Auditors for HTTPS Clients . . . . . . . . . 28 | 10.5.3. Privacy for HTTPS clients performing STH Proof | |||
10.4.7. HTTPS Clients as Auditors . . . . . . . . . . . . . 28 | Fetching . . . . . . . . . . . . . . . . . . . . . . 28 | |||
11. Policy Recommendations . . . . . . . . . . . . . . . . . . . 29 | 10.5.4. Privacy in STH Pollination . . . . . . . . . . . . . 28 | |||
11.1. Blocking Recommendations . . . . . . . . . . . . . . . . 29 | 10.5.5. Privacy in STH Interaction . . . . . . . . . . . . . 29 | |||
11.1.1. Frustrating blocking . . . . . . . . . . . . . . . . 29 | 10.5.6. Trusted Auditors for HTTPS Clients . . . . . . . . . 29 | |||
11.1.2. Responding to possible blocking . . . . . . . . . . 29 | 10.5.7. HTTPS Clients as Auditors . . . . . . . . . . . . . 30 | |||
11.2. Proof Fetching Recommendations . . . . . . . . . . . . . 31 | 11. Policy Recommendations . . . . . . . . . . . . . . . . . . . 30 | |||
11.3. Record Distribution Recommendations . . . . . . . . . . 31 | 11.1. Blocking Recommendations . . . . . . . . . . . . . . . . 31 | |||
11.3.1. Mixing Algorithm . . . . . . . . . . . . . . . . . . 32 | 11.1.1. Frustrating blocking . . . . . . . . . . . . . . . . 31 | |||
11.3.2. Flushing Attacks . . . . . . . . . . . . . . . . . . 33 | 11.1.2. Responding to possible blocking . . . . . . . . . . 31 | |||
11.3.3. The Deletion Algorithm . . . . . . . . . . . . . . . 34 | 11.2. Proof Fetching Recommendations . . . . . . . . . . . . . 32 | |||
12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 45 | 11.3. Record Distribution Recommendations . . . . . . . . . . 33 | |||
13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 45 | 11.3.1. Mixing Algorithm . . . . . . . . . . . . . . . . . . 34 | |||
14. ChangeLog . . . . . . . . . . . . . . . . . . . . . . . . . . 45 | 11.3.2. The Deletion Algorithm . . . . . . . . . . . . . . . 35 | |||
14.1. Changes between ietf-01 and ietf-02 . . . . . . . . . . 45 | 11.4. Concrete Recommendations . . . . . . . . . . . . . . . . 36 | |||
14.2. Changes between ietf-00 and ietf-01 . . . . . . . . . . 46 | 11.4.1. STH Pollination . . . . . . . . . . . . . . . . . . 36 | |||
14.3. Changes between -01 and -02 . . . . . . . . . . . . . . 46 | 11.4.2. SCT Feedback . . . . . . . . . . . . . . . . . . . . 39 | |||
14.4. Changes between -00 and -01 . . . . . . . . . . . . . . 46 | 12. IANA considerations . . . . . . . . . . . . . . . . . . . . . 53 | |||
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 47 | 13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 53 | |||
15.1. Normative References . . . . . . . . . . . . . . . . . . 47 | 14. ChangeLog . . . . . . . . . . . . . . . . . . . . . . . . . . 53 | |||
15.2. Informative References . . . . . . . . . . . . . . . . . 47 | 14.1. Changes between ietf-02 and ietf-03 . . . . . . . . . . 53 | |||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 47 | 14.2. Changes between ietf-01 and ietf-02 . . . . . . . . . . 54 | |||
14.3. Changes between ietf-00 and ietf-01 . . . . . . . . . . 54 | ||||
14.4. Changes between -01 and -02 . . . . . . . . . . . . . . 54 | ||||
14.5. Changes between -00 and -01 . . . . . . . . . . . . . . 55 | ||||
15. References . . . . . . . . . . . . . . . . . . . . . . . . . 55 | ||||
15.1. Normative References . . . . . . . . . . . . . . . . . . 55 | ||||
15.2. Informative References . . . . . . . . . . . . . . . . . 55 | ||||
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 56 | ||||
1. Introduction | 1. Introduction | |||
The purpose of the protocols in this document, collectively referred | The purpose of the protocols in this document, collectively referred | |||
to as CT Gossip, is to detect certain misbehavior by CT logs. In | to as CT Gossip, is to detect certain misbehavior by CT logs. In | |||
particular, CT Gossip aims to detect logs that are providing | particular, CT Gossip aims to detect logs that are providing | |||
inconsistent views to different log clients, and logs failing to | inconsistent views to different log clients, and logs failing to | |||
include submitted certificates within the time period stipulated by | include submitted certificates within the time period stipulated by | |||
MMD. | MMD. | |||
[ TODO: enumerate the interfaces used for detecting misbehaviour? ] | ||||
One of the major challenges of any gossip protocol is limiting damage | One of the major challenges of any gossip protocol is limiting damage | |||
to user privacy. The goal of CT gossip is to publish and distribute | to user privacy. The goal of CT gossip is to publish and distribute | |||
information about the logs and their operations, but not to expose | information about the logs and their operations, but not to expose | |||
any additional information about the operation of any of the other | any additional information about the operation of any of the other | |||
participants. Privacy of consumers of log information (in | participants. Privacy of consumers of log information (in | |||
particular, of web browsers and other TLS clients) should not be | particular, of web browsers and other TLS clients) should not be | |||
undermined by gossip. | undermined by gossip. | |||
This document presents three different, complementary mechanisms for | This document presents three different, complementary mechanisms for | |||
non-log elements of the CT ecosystem to exchange information about | non-log elements of the CT ecosystem to exchange information about | |||
skipping to change at page 4, line 18 ¶ | skipping to change at page 4, line 20 ¶ | |||
this is described as a partitioning attack. Each client would be | this is described as a partitioning attack. Each client would be | |||
able to verify the append-only nature of the log but, in the extreme | able to verify the append-only nature of the log but, in the extreme | |||
case, each client might see a unique view of the log. | case, each client might see a unique view of the log. | |||
The CT logs are public, append-only and untrusted and thus have to be | The CT logs are public, append-only and untrusted and thus have to be | |||
audited for consistency, i.e., they should never rewrite history. | audited for consistency, i.e., they should never rewrite history. | |||
Additionally, auditors and other log clients need to exchange | Additionally, auditors and other log clients need to exchange | |||
information about logs in order to be able to detect a partitioning | information about logs in order to be able to detect a partitioning | |||
attack (as described above). | attack (as described above). | |||
Gossiping about log behaviour helps address the problem of detecting | Gossiping about log behavior helps address the problem of detecting | |||
malicious or compromised logs with respect to a partitioning attack. | malicious or compromised logs with respect to a partitioning attack. | |||
We want some side of the partitioned tree, and ideally both sides, to | We want some side of the partitioned tree, and ideally both sides, to | |||
see the other side. | see the other side. | |||
Disseminating information about a log poses a potential threat to the | Disseminating information about a log poses a potential threat to the | |||
privacy of end users. Some data of interest (e.g. SCTs) is linkable | privacy of end users. Some data of interest (e.g. SCTs) is linkable | |||
to specific log entries and thereby to specific websites, which makes | to specific log entries and thereby to specific websites, which makes | |||
sharing them with others a privacy concern. Gossiping about this | sharing them with others a privacy concern. Gossiping about this | |||
data has to take privacy considerations into account in order not to | data has to take privacy considerations into account in order not to | |||
expose associations between users of the log (e.g., web browsers) and | expose associations between users of the log (e.g., web browsers) and | |||
certificate holders (e.g., web sites). Even sharing STHs (which do | certificate holders (e.g., web sites). Even sharing STHs (which do | |||
not link to specific log entries) can be problematic - user tracking | not link to specific log entries) can be problematic - user tracking | |||
by fingerprinting through rare STHs is one potential attack (see | by fingerprinting through rare STHs is one potential attack (see | |||
Section 8.2). | Section 8.2). | |||
3. Overview | 3. Overview | |||
This document presents three gossiping mechanisms: SCT Feedback, STH | ||||
Pollination, and a Trusted Auditor Relationship. | ||||
SCT Feedback enables HTTPS clients to share Signed Certificate | SCT Feedback enables HTTPS clients to share Signed Certificate | |||
Timestamps (SCTs) (Section 3.3 of [RFC-6962-BIS-09]) with CT auditors | Timestamps (SCTs) (Section 3.3 of [RFC-6962-BIS-09]) with CT auditors | |||
in a privacy-preserving manner by sending SCTs to originating HTTPS | in a privacy-preserving manner by sending SCTs to originating HTTPS | |||
servers, who in turn share them with CT auditors. | servers, who in turn share them with CT auditors. | |||
In STH Pollination, HTTPS clients use HTTPS servers as pools to share | In STH Pollination, HTTPS clients use HTTPS servers as pools to share | |||
Signed Tree Heads (STHs) (Section 3.6 of [RFC-6962-BIS-09]) with | Signed Tree Heads (STHs) (Section 3.6 of [RFC-6962-BIS-09]) with | |||
other connecting clients in the hope that STHs will find their way to | other connecting clients in the hope that STHs will find their way to | |||
CT auditors. | CT auditors. | |||
skipping to change at page 5, line 9 ¶ | skipping to change at page 5, line 17 ¶ | |||
sensitive data being handled according to whatever privacy policy is | sensitive data being handled according to whatever privacy policy is | |||
agreed on between client and trusted party. | agreed on between client and trusted party. | |||
Despite the privacy risks with sharing SCTs there is no loss in | Despite the privacy risks with sharing SCTs there is no loss in | |||
privacy if a client sends SCTs for a given site to the site | privacy if a client sends SCTs for a given site to the site | |||
corresponding to the SCT. This is because the site's logs would | corresponding to the SCT. This is because the site's logs would | |||
already indicate that the client is accessing that site. In this way | already indicate that the client is accessing that site. In this way | |||
a site can accumulate records of SCTs that have been issued by | a site can accumulate records of SCTs that have been issued by | |||
various logs for that site, providing a consolidated repository of | various logs for that site, providing a consolidated repository of | |||
SCTs that could be shared with auditors. Auditors can use this | SCTs that could be shared with auditors. Auditors can use this | |||
information to detect logs that misbehave by not including | information to detect a misbehaving log that fails to include a | |||
certificates within the time period stipulated by the MMD metadata. | certificate within the time period stipulated by its MMD metadata. | |||
Sharing an STH is considered reasonably safe from a privacy | Sharing an STH is considered reasonably safe from a privacy | |||
perspective as long as the same STH is shared by a large number of | perspective as long as the same STH is shared by a large number of | |||
other log clients. This safety in numbers can be achieved by only | other log clients. This safety in numbers can be achieved by only | |||
allowing gossiping of STHs issued in a certain window of time, while | allowing gossiping of STHs issued in a certain window of time, while | |||
also refusing to gossip about STHs from logs with too high an STH | also refusing to gossip about STHs from logs with too high an STH | |||
issuance frequency (see Section 8.2). | issuance frequency (see Section 8.2). | |||
4. Terminology | 4. Terminology | |||
This document relies on terminology and data structures defined in | This document relies on terminology and data structures defined in | |||
[RFC-6962-BIS-09], including STH, SCT, Version, LogID, SCT timestamp, | [RFC-6962-BIS-09], including MMD, STH, SCT, Version, LogID, SCT | |||
CtExtensions, SCT signature, Merkle Tree Hash. | timestamp, CtExtensions, SCT signature, Merkle Tree Hash. | |||
This document relies on terminology defined in | This document relies on terminology defined in | |||
[draft-ietf-trans-threat-analysis-03], including Auditing. | [draft-ietf-trans-threat-analysis-03], including Auditing. | |||
4.1. Pre-Loaded vs Locally Added Anchors | 4.1. Pre-Loaded vs Locally Added Anchors | |||
Through the document, we refer to both Trust Anchors (Certificate | Through the document, we refer to both Trust Anchors (Certificate | |||
Authorities) and Logs. Both Logs and Trust Anchors may be locally | Authorities) and Logs. Both Logs and Trust Anchors may be locally | |||
added by an administrator. Unless otherwise clarified, in both cases | added by an administrator. Unless otherwise clarified, in both cases | |||
we refer to the set of Trust Anchors and Logs that come pre-loaded | we refer to the set of Trust Anchors and Logs that come pre-loaded | |||
skipping to change at page 6, line 18 ¶ | skipping to change at page 6, line 22 ¶ | |||
o SCT Feedback - transporting SCTs and certificate chains from HTTPS | o SCT Feedback - transporting SCTs and certificate chains from HTTPS | |||
clients to CT auditors via HTTPS servers. | clients to CT auditors via HTTPS servers. | |||
o STH Pollination - HTTPS clients and CT auditors using HTTPS | o STH Pollination - HTTPS clients and CT auditors using HTTPS | |||
servers as STH pools for exchanging STHs. | servers as STH pools for exchanging STHs. | |||
o Trusted Auditor Stream - HTTPS clients communicating directly with | o Trusted Auditor Stream - HTTPS clients communicating directly with | |||
trusted CT auditors sharing SCTs, certificate chains and STHs. | trusted CT auditors sharing SCTs, certificate chains and STHs. | |||
It is worthwhile to note that when an HTTPS Client or CT auditor | It is worthwhile to note that when an HTTPS client or CT auditor | |||
interact with a log, they may equivalently interact with a log mirror | interacts with a log, they may equivalently interact with a log | |||
or cache that replicates the log. | mirror or cache that replicates the log. | |||
7. Data flow | 7. Data flow | |||
The following picture shows how certificates, SCTs and STHs flow | The following picture shows how certificates, SCTs and STHs flow | |||
through a CT system with SCT Feedback and STH Pollination. It does | through a CT system with SCT Feedback and STH Pollination. It does | |||
not show what goes in the Trusted Auditor Relationship stream. | not show what goes in the Trusted Auditor Relationship stream. | |||
+- Cert ---- +----------+ | +- Cert ---- +----------+ | |||
| | CA | ----------+ | | | CA | ----------+ | |||
| + SCT -> +----------+ | | | + SCT -> +----------+ | | |||
v | Cert [& SCT] | v | Cert [& SCT] | |||
+----------+ | | +----------+ | | |||
| Log | ---------- SCT -----------+ | | Log | ---------- SCT -----------+ | |||
+----------+ v | +----------+ v | |||
| ^ +----------+ | | ^ +----------+ | |||
| | SCT & Certs --- | Website | | | | SCTs & Certs --- | Website | | |||
| |[1] | +----------+ | | |[1] | +----------+ | |||
| |[2] STH ^ | | | |[2] STHs ^ | | |||
| |[3] v | | | | |[3] v | | | |||
| | +----------+ | | | | | +----------+ | | | |||
| +--------> | Auditor | | HTTPS traffic | | +--------> | Auditor | | HTTPS traffic | |||
| +----------+ | | | | +----------+ | | | |||
STH | SCT | STH | SCT & Cert | |||
| SCT & Certs | | | SCTs & Certs | | |||
Log entries | | | Log entries | | | |||
| STH STH | | STHs STHs | |||
v | | | v | | | |||
+----------+ | v | +----------+ | v | |||
| Monitor | +----------+ | | Monitor | +----------+ | |||
+----------+ | Browser | | +----------+ | Browser | | |||
+----------+ | +----------+ | |||
# Auditor Log | # Auditor Log | |||
[1] |--- get-sth ------------------->| | [1] |--- get-sth ------------------->| | |||
|<-- STH ------------------------| | |<-- STH ------------------------| | |||
[2] |--- leaf hash + tree size ----->| | [2] |--- leaf hash + tree size ----->| | |||
skipping to change at page 7, line 50 ¶ | skipping to change at page 7, line 50 ¶ | |||
8.1. SCT Feedback | 8.1. SCT Feedback | |||
The goal of SCT Feedback is for clients to share SCTs and certificate | The goal of SCT Feedback is for clients to share SCTs and certificate | |||
chains with CT auditors while still preserving the privacy of the end | chains with CT auditors while still preserving the privacy of the end | |||
user. The sharing of SCTs contribute to the overall goal of | user. The sharing of SCTs contribute to the overall goal of | |||
detecting misbehaving logs by providing auditors with SCTs from many | detecting misbehaving logs by providing auditors with SCTs from many | |||
vantage points, making it more likely to catch a violation of a log's | vantage points, making it more likely to catch a violation of a log's | |||
MMD or a log presenting inconsistent views. The sharing of | MMD or a log presenting inconsistent views. The sharing of | |||
certificate chains is beneficial to HTTPS server operators interested | certificate chains is beneficial to HTTPS server operators interested | |||
in direct feedback from clients for detecting bogus certificates | in direct feedback from clients for detecting bogus certificates | |||
issued in their name and therefore incentivises server operators to | issued in their name and therefore incentivizes server operators to | |||
take part in SCT Feedback. | take part in SCT Feedback. | |||
SCT Feedback is the most privacy-preserving gossip mechanism, as it | SCT Feedback is the most privacy-preserving gossip mechanism, as it | |||
does not directly expose any links between an end user and the sites | does not directly expose any links between an end user and the sites | |||
they've visisted to any third party. | they've visited to any third party. | |||
HTTPS clients store SCTs and certificate chains they see, and later | HTTPS clients store SCTs and certificate chains they see, and later | |||
send them to the originating HTTPS server by posting them to a well- | send them to the originating HTTPS server by posting them to a well- | |||
known URL (associated with that server), as described in | known URL (associated with that server), as described in | |||
Section 8.1.2. Note that clients will send the same SCTs and chains | Section 8.1.2. Note that clients will send the same SCTs and chains | |||
to a server multiple times with the assumption that any man-in-the- | to a server multiple times with the assumption that any man-in-the- | |||
middle attack eventually will cease, and an honest server will | middle attack eventually will cease, and an honest server will | |||
eventually receive collected malicious SCTs and certificate chains. | eventually receive collected malicious SCTs and certificate chains. | |||
HTTPS servers store SCTs and certificate chains received from | HTTPS servers store SCTs and certificate chains received from | |||
clients, as described in Section 8.1.3. They later share them with | clients, as described in Section 8.1.3. They later share them with | |||
CT auditors by either posting them to auditors or making them | CT auditors by either posting them to auditors or making them | |||
available via a well-known URL. This is described in Section 8.1.4. | available via a well-known URL. This is described in Section 8.1.4. | |||
8.1.1. SCT Feedback data format | 8.1.1. SCT Feedback data format | |||
The data shared between HTTPS clients and servers, as well as between | The data shared between HTTPS clients and servers, as well as between | |||
HTTPS servers and CT auditors, is a JSON array [RFC7159]. Each item | HTTPS servers and CT auditors, is a JSON array [RFC7159]. Each item | |||
in the array is a JSON object with the following content: | in the array is a JSON object with the following content: | |||
o x509_chain: An array of base64-encoded X.509 certificates. The | o x509_chain: An array of PEM-encoded X.509 certificates. The first | |||
first element is the end-entity certificate, the second certifies | element is the end-entity certificate, the second certifies the | |||
the first and so on. | first and so on. | |||
o sct_data: An array of objects consisting of the base64 | o sct_data: An array of objects consisting of the base64 | |||
representation of the binary SCT data as defined in | representation of the binary SCT data as defined in | |||
[RFC-6962-BIS-09] Section 3.3. | [RFC-6962-BIS-09] Section 3.3. | |||
We will refer to this object as 'sct_feedback'. | We will refer to this object as 'sct_feedback'. | |||
The x509_chain element always contains at least one element. It also | The x509_chain element always contains a full chain from a leaf | |||
always contains a full chain from a leaf certificate to a self-signed | certificate to a self-signed trust anchor. | |||
trust anchor. | ||||
[ TBD: Be strict about what sct_data may contain or is this | See Section 8.1.2 for details on what the sct_data element contains | |||
sufficiently implied by previous sections? ] | as well as more details about the x509_chain element. | |||
8.1.2. HTTPS client to server | 8.1.2. HTTPS client to server | |||
When an HTTPS client connects to an HTTPS server, the client receives | When an HTTPS client connects to an HTTPS server, the client receives | |||
a set of SCTs as part of the TLS handshake. SCTs are included in the | a set of SCTs as part of the TLS handshake. SCTs are included in the | |||
TLS handshake using one or more of the three mechanisms described in | TLS handshake using one or more of the three mechanisms described in | |||
[RFC-6962-BIS-09] section 3.4 - in the server certificate, in a TLS | [RFC-6962-BIS-09] section 3.4 - in the server certificate, in a TLS | |||
extension, or in an OCSP extension. The client MUST discard SCTs | extension, or in an OCSP extension. The client MUST discard SCTs | |||
that are not signed by a log known to the client and SHOULD store the | that are not signed by a log known to the client and SHOULD store the | |||
remaining SCTs together with a locally constructed certificate chain | remaining SCTs together with a locally constructed certificate chain | |||
skipping to change at page 9, line 31 ¶ | skipping to change at page 9, line 30 ¶ | |||
auditors receiving SCTs from the HTTPS server would learn information | auditors receiving SCTs from the HTTPS server would learn information | |||
about other HTTPS servers visited by its clients. | about other HTTPS servers visited by its clients. | |||
If the client later again connects to the same HTTPS server, it again | If the client later again connects to the same HTTPS server, it again | |||
receives a set of SCTs and calculates a certificate chain, and again | receives a set of SCTs and calculates a certificate chain, and again | |||
creates an sct_feedback or similar object. If this object does not | creates an sct_feedback or similar object. If this object does not | |||
exactly match an existing object in the store, then the client MUST | exactly match an existing object in the store, then the client MUST | |||
add this new object to the store, associated with the exact domain | add this new object to the store, associated with the exact domain | |||
name contacted, as described above. An exact comparison is needed to | name contacted, as described above. An exact comparison is needed to | |||
ensure that attacks involving alternate chains are detected. An | ensure that attacks involving alternate chains are detected. An | |||
example of such an attack is described in [TODO double-CA-compromise | example of such an attack is described in | |||
attack]. However, at least one optimization is safe and MAY be | [dual-ca-compromise-attack]. However, at least one optimization is | |||
performed: If the certificate chain exactly matches an existing | safe and MAY be performed: If the certificate chain exactly matches | |||
certificate chain, the client may store the union of the SCTs from | an existing certificate chain, the client MAY store the union of the | |||
the two objects in the first (existing) object. | SCTs from the two objects in the first (existing) object. | |||
If the client does connect to the same HTTPS server a subsequent | If the client does connect to the same HTTPS server a subsequent | |||
time, it MUST send to the server sct_feedback objects in the store | time, it MUST send to the server sct_feedback objects in the store | |||
that are associated with that domain name. It is not necessary to | that are associated with that domain name. However, it is not | |||
send an sct_feedback object constructed from the current TLS session. | necessary to send an sct_feedback object constructed from the current | |||
TLS session, and if the client does so, it MUST NOT be marked as sent | ||||
The client MUST NOT send the same set of SCTs to the same server more | in any internal tracking done by the client. | |||
often than TBD. | ||||
[ TODO: expand on rate/resource limiting motivation ] | ||||
Refer to Section 11.3 for recommendations about strategies. | Refer to Section 11.3 for recommendations for implementation. | |||
Because SCTs can be used as a tracking mechanism (see | Because SCTs can be used as a tracking mechanism (see | |||
Section 10.4.2), they deserve special treatment when they are | Section 10.5.2), they deserve special treatment when they are | |||
received from (and provided to) domains that are loaded as | received from (and provided to) domains that are loaded as | |||
subresources from an origin domain. Such domains are commonly called | subresources from an origin domain. Such domains are commonly called | |||
'third party domains'. An HTTPS Client SHOULD store SCT Feedback | 'third party domains'. An HTTPS client SHOULD store SCT Feedback | |||
using a 'double-keying' approach, which isolates third party domains | using a 'double-keying' approach, which isolates third party domains | |||
by the first party domain. This is described in XXX. Gossip would | by the first party domain. This is described in [double-keying]. | |||
be performed normally for third party domains only when the user | ||||
revisits the first party domain. In lieu of 'double-keying', an | ||||
HTTPS Client MAY treat SCT Feedback in the same manner it treats | ||||
other security mechanisms that can enable tracking (such as HSTS and | ||||
HPKP.) | ||||
[ XXX is currently https://www.torproject.org/projects/torbrowser/ | Gossip would be performed normally for third party domains only when | |||
design/#identifier-linkability How should it be references? Do we | the user revisits the first party domain. In lieu of 'double- | |||
need to copy this out into another document? An appendix? ] | keying', an HTTPS client MAY treat SCT Feedback in the same manner it | |||
treats other security mechanisms that can enable tracking (such as | ||||
HSTS and HPKP.) | ||||
If the HTTPS client has configuration options for not sending cookies | If the HTTPS client has configuration options for not sending cookies | |||
to third parties, SCTs of third parties MUST be treated as cookies | to third parties, SCTs of third parties MUST be treated as cookies | |||
with respect to this setting. This prevents third party tracking | with respect to this setting. This prevents third party tracking | |||
through the use of SCTs/certificates, which would bypass the cookie | through the use of SCTs/certificates, which would bypass the cookie | |||
policy. | policy. For domains that are only loaded as third party domains, the | |||
client may never perform SCT Feedback; however the client may perform | ||||
STH Pollination after fetching an inclusion proof, as specified in | ||||
Section 8.2. | ||||
SCTs and corresponding certificates are POSTed to the originating | SCTs and corresponding certificates are POSTed to the originating | |||
HTTPS server at the well-known URL: | HTTPS server at the well-known URL: | |||
https://<domain>/.well-known/ct-gossip/v1/sct-feedback | https://<domain>/.well-known/ct-gossip/v1/sct-feedback | |||
The data sent in the POST is defined in Section 8.1.1. This data | The data sent in the POST is defined in Section 8.1.1. This data | |||
SHOULD be sent in an already established TLS session. This makes it | SHOULD be sent in an already-established TLS session. This makes it | |||
hard for an attacker to disrupt SCT Feedback without also disturbing | hard for an attacker to disrupt SCT Feedback without also disturbing | |||
ordinary secure browsing (https://). This is discussed more in | ordinary secure browsing (https://). This is discussed more in | |||
Section 11.1.1. | Section 11.1.1. | |||
The HTTPS server SHOULD respond with an HTTP 200 response code and an | ||||
empty body if it was able to process the request. An HTTPS client | ||||
who receives any other response SHOULD consider it an error. | ||||
Some clients have trust anchors or logs that are locally added (e.g. | Some clients have trust anchors or logs that are locally added (e.g. | |||
by an administrator or by the user themselves). These additions are | by an administrator or by the user themselves). These additions are | |||
potentially privacy-sensitive because they can carry information | potentially privacy-sensitive because they can carry information | |||
about the specific configuration, computer, or user. | about the specific configuration, computer, or user. | |||
Certificates validated by locally added trust anchors will commonly | Certificates validated by locally added trust anchors will commonly | |||
have no SCTs associated with them, so in this case no action is | have no SCTs associated with them, so in this case no action is | |||
needed with respect to CT Gossip. SCTs issued by locally added logs | needed with respect to CT Gossip. SCTs issued by locally added logs | |||
MUST NOT be reported via SCT Feedback. | MUST NOT be reported via SCT Feedback. | |||
If a certificate is validated by SCTs that are issued by publicly | If a certificate is validated by SCTs that are issued by publicly | |||
trusted logs, but chains to a local trust anchor, the client MAY | trusted logs, but chains to a local trust anchor, the client MAY | |||
perfom SCT Feedback for this SCT and certificate chain bundle. If it | perform SCT Feedback for this SCT and certificate chain bundle. If | |||
does so, the client MUST include the full chain of certificates | it does so, the client MUST include the full chain of certificates | |||
chaining to the local trust anchor in the x509_chain array. | chaining to the local trust anchor in the x509_chain array. | |||
Perfoming SCT Feedback in this scenario may be advantageous for the | Performing SCT Feedback in this scenario may be advantageous for the | |||
broader internet and CT ecosystem, but may also disclose information | broader internet and CT ecosystem, but may also disclose information | |||
about the client. If the client elects to omit SCT Feedback, it can | about the client. If the client elects to omit SCT Feedback, it can | |||
still choose to perform STH Pollination after fetching an inclusion | choose to perform STH Pollination after fetching an inclusion proof, | |||
proof, as specified in Section 8.2. | as specified in Section 8.2. | |||
We require the client to send the full chain (or nothing at all) for | We require the client to send the full chain (or nothing at all) for | |||
two reasons. Firstly, it simplifies the operation on the server if | two reasons. Firstly, it simplifies the operation on the server if | |||
there are not two code paths. Secondly, omitting the chain does not | there are not two code paths. Secondly, omitting the chain does not | |||
actually preserve user privacy. The Issuer field in the certificate | actually preserve user privacy. The Issuer field in the certificate | |||
describes the signing certificate. And if the certificate is being | describes the signing certificate. And if the certificate is being | |||
submitted at all, it means the certificate is logged, and has SCTs. | submitted at all, it means the certificate is logged, and has SCTs. | |||
This means that the Issuer can be queried and obtained from the log | This means that the Issuer can be queried and obtained from the log, | |||
so omitting the parent from the client's submission does not actually | so omitting the signing certificate from the client's submission does | |||
help user privacy. | not actually help user privacy. | |||
8.1.3. HTTPS server operation | 8.1.3. HTTPS server operation | |||
HTTPS servers can be configured (or omit configuration), resulting | HTTPS servers can be configured (or omit configuration), resulting | |||
in, broadly, two modes of operation. In the simpler mode, the server | in, broadly, two modes of operation. In the simpler mode, the server | |||
will only track leaf certificates and SCTs applicable to those leaf | will only track leaf certificates and SCTs applicable to those leaf | |||
certificates. In the more complex mode, the server will confirm the | certificates. In the more complex mode, the server will confirm the | |||
client's chain validation and store the certificate chain. The | client's chain validation and store the certificate chain. The | |||
latter mode requires more configuration, but is necessary to prevent | latter mode requires more configuration, but is necessary to prevent | |||
denial of service (DoS) attacks on the server's storage space. | denial of service (DoS) attacks on the server's storage space. | |||
In the simple mode of operation, upon recieving a submission at the | In the simple mode of operation, upon receiving a submission at the | |||
sct-feedback well-known URL, an HTTPS server will perform a set of | sct-feedback well-known URL, an HTTPS server will perform a set of | |||
operations, checking on each sct_feedback object before storing it: | operations, checking on each sct_feedback object before storing it: | |||
1. the HTTPS server MAY modify the sct_feedback object, and discard | 1. the HTTPS server MAY modify the sct_feedback object, and discard | |||
all items in the x509_chain array except the first item (which is | all items in the x509_chain array except the first item (which is | |||
the end-entity certificate) | the end-entity certificate) | |||
2. if a bit-wise compare of the sct_feedback object matches one | 2. if a bit-wise compare of the sct_feedback object matches one | |||
already in the store, this sct_feedback object SHOULD be | already in the store, this sct_feedback object SHOULD be | |||
discarded | discarded | |||
skipping to change at page 12, line 6 ¶ | skipping to change at page 12, line 5 ¶ | |||
SCT for the accompanying leaf cert, and issued by a known log, | SCT for the accompanying leaf cert, and issued by a known log, | |||
the individual SCT SHOULD be discarded | the individual SCT SHOULD be discarded | |||
The modification in step number 1 is necessary to prevent a malicious | The modification in step number 1 is necessary to prevent a malicious | |||
client from exhausting the server's storage space. A client can | client from exhausting the server's storage space. A client can | |||
generate their own issuing certificate authorities, and create an | generate their own issuing certificate authorities, and create an | |||
arbitrary number of chains that terminate in an end-entity | arbitrary number of chains that terminate in an end-entity | |||
certificate with an existing SCT. By discarding all but the end- | certificate with an existing SCT. By discarding all but the end- | |||
entity certificate, we prevent a simple HTTPS server from storing | entity certificate, we prevent a simple HTTPS server from storing | |||
this data. Note that operation in this mode will not prevent the | this data. Note that operation in this mode will not prevent the | |||
attack described in Section 10.2. Skipping this step requires | attack described in [dual-ca-compromise-attack]. Skipping this step | |||
additional configuration as described below. | requires additional configuration as described below. | |||
The check in step 2 is for detecting duplicates and minimizing | The check in step 2 is for detecting duplicates and minimizing | |||
processing and storage by the server. As on the client, an exact | processing and storage by the server. As on the client, an exact | |||
comparison is needed to ensure that attacks involving alternate | comparison is needed to ensure that attacks involving alternate | |||
chains are detected. Again, at least one optimization is safe and | chains are detected. Again, at least one optimization is safe and | |||
MAY be performed. If the certificate chain exactly matches an | MAY be performed. If the certificate chain exactly matches an | |||
existing certificate chain, the server may store the union of the | existing certificate chain, the server MAY store the union of the | |||
SCTs from the two objects in the first (existing) object. It should | SCTs from the two objects in the first (existing) object. If the | |||
do this after completing the validity check on the SCTs. | validity check on any of the SCTs fails, the server SHOULD NOT store | |||
the union of the SCTs. | ||||
The check in step 3 is to help malfunctioning clients from exposing | The check in step 3 is to help malfunctioning clients from exposing | |||
which sites they visit. It additionally helps prevent DoS attacks on | which sites they visit. It additionally helps prevent DoS attacks on | |||
the server. | the server. | |||
[ TBD: Thinking about building this, how does the SCT Feedback app | [ Note: Thinking about building this, how does the SCT Feedback app | |||
know which sites it's authoritative for? ] | know which sites it's authoritative for? It will need that amount of | |||
configuration at least. ] | ||||
The check in step 4 is to prevent DoS attacks where an adversary | The check in step 4 is to prevent DoS attacks where an adversary | |||
fills up the store prior to attacking a client (thus preventing the | fills up the store prior to attacking a client (thus preventing the | |||
client's feedback from being recorded), or an attack where an | client's feedback from being recorded), or an attack where an | |||
adversary simply attempts to fill up server's storage space. | adversary simply attempts to fill up server's storage space. | |||
The more advanced server configuration will detect the [TODO double- | The above describes the simpler mode of operation. In the more | |||
CA-compromise] attack. In this configuration the server will not | advanced server mode, the server will detect the attack described in | |||
modify the sct_feedback object prior to performing checks 2, 3, and | [dual-ca-compromise-attack]. In this configuration the server will | |||
4. | not modify the sct_feedback object prior to performing checks 2, 3, | |||
and 4. | ||||
To prevent a malicious client from filling the server's data store, | To prevent a malicious client from filling the server's data store, | |||
the HTTPS Server SHOULD perform an additional check: | the HTTPS server SHOULD perform an additional check in the more | |||
advanced mode: | ||||
1. if the x509_chain consists of an invalid certificate chain, or | o if the x509_chain consists of an invalid certificate chain, or the | |||
the culminating trust anchor is not recognized by the server, the | culminating trust anchor is not recognized by the server, the | |||
server SHOULD modify the sct_feedback object, discarding all | server SHOULD modify the sct_feedback object, discarding all items | |||
items in the x509_chain array except the first item | in the x509_chain array except the first item | |||
The HTTPS server may choose to omit checks 4 or 5. This will place | The HTTPS server MAY choose to omit checks 4 or 5. This will place | |||
the server at risk of having its data store filled up by invalid | the server at risk of having its data store filled up by invalid | |||
data, but can also allow a server to identify interesting certificate | data, but can also allow a server to identify interesting certificate | |||
or certificate chains that omit valid SCTs, or do not chain to a | or certificate chains that omit valid SCTs, or do not chain to a | |||
trusted root. This information may enable an HTTPS server operator | trusted root. This information may enable an HTTPS server operator | |||
to detect attacks or unusual behavior of Certificate Authorities even | to detect attacks or unusual behavior of Certificate Authorities even | |||
outside the Certificate Transparency ecosystem. | outside the Certificate Transparency ecosystem. | |||
8.1.4. HTTPS server to auditors | 8.1.4. HTTPS server to auditors | |||
HTTPS servers receiving SCTs from clients SHOULD share SCTs and | HTTPS servers receiving SCTs from clients SHOULD share SCTs and | |||
certificate chains with CT auditors by either serving them on the | certificate chains with CT auditors by either serving them on the | |||
well-known URL: | well-known URL: | |||
https://<domain>/.well-known/ct-gossip/v1/collected-sct-feedback | https://<domain>/.well-known/ct-gossip/v1/collected-sct-feedback | |||
or by HTTPS POSTing them to a set of preconfigured auditors. This | or by HTTPS POSTing them to a set of preconfigured auditors. This | |||
allows an HTTPS server to choose between an active push model or a | allows an HTTPS server to choose between an active push model or a | |||
passive pull model. | passive pull model. | |||
The data received in a GET of the well-known URL or sent in the POST | The data received in a GET of the well-known URL or sent in the POST | |||
is defined in Section 8.1.1. | is defined in Section 8.1.1 with the following difference: The | |||
x509_chain element may contain only he end-entity certificate, as | ||||
described below. | ||||
HTTPS servers SHOULD share all sct_feedback objects they see that | HTTPS servers SHOULD share all sct_feedback objects they see that | |||
pass the checks in Section 8.1.3. If this is an infeasible amount of | pass the checks in Section 8.1.3. If this is an infeasible amount of | |||
data, the server may choose to expire submissions according to an | data, the server MAY choose to expire submissions according to an | |||
undefined policy. Suggestions for such a policy can be found in | undefined policy. Suggestions for such a policy can be found in | |||
Section 11.3. | Section 11.3. | |||
HTTPS servers MUST NOT share any other data that they may learn from | HTTPS servers MUST NOT share any other data that they may learn from | |||
the submission of SCT Feedback by HTTPS clients, like the HTTPS | the submission of SCT Feedback by HTTPS clients, like the HTTPS | |||
client IP address or the time of submission. | client IP address or the time of submission. | |||
As described above, HTTPS servers can be configured (or omit | As described above, HTTPS servers can be configured (or omit | |||
configuration), resulting in two modes of operation. In one mode, | configuration), resulting in two modes of operation. In one mode, | |||
the x509_chain array will contain a full certificate chain. This | the x509_chain array will contain a full certificate chain. This | |||
chain may terminate in a trust anchor the auditor may recognize, or | chain may terminate in a trust anchor the auditor may recognize, or | |||
it may not. (One scenario where this could occur is if the client | it may not. (One scenario where this could occur is if the client | |||
submitted a chain terminiating in a locally added trust anchor, and | submitted a chain terminating in a locally added trust anchor, and | |||
the server kept this chain.) In the other mode, the x509_chain array | the server kept this chain.) In the other mode, the x509_chain array | |||
will consist of only a single element, which is the end-entity | will consist of only a single element, which is the end-entity | |||
certificate. | certificate. | |||
Auditors SHOULD provide the following URL accepting HTTPS POSTing of | Auditors SHOULD provide the following URL accepting HTTPS POSTing of | |||
SCT feedback data: | SCT feedback data: | |||
https://<auditor>/ct-gossip/v1/sct-feedback | https://<auditor>/ct-gossip/v1/sct-feedback | |||
[ TBD: Should that be .well-known? Depends on whether auditors will | ||||
operate in their own URL name space or not. ] | ||||
Auditors SHOULD regularly poll HTTPS servers at the well-known | Auditors SHOULD regularly poll HTTPS servers at the well-known | |||
collected-sct-feedback URL. The frequency of the polling and how to | collected-sct-feedback URL. The frequency of the polling and how to | |||
determine which domains to poll is outside the scope of this | determine which domains to poll is outside the scope of this | |||
document. However, the selection MUST NOT be influenced by potential | document. However, the selection MUST NOT be influenced by potential | |||
HTTPS clients connecting directly to the auditor. For example, if a | HTTPS clients connecting directly to the auditor. For example, if a | |||
poll to example.com occurs directly after a client submits an SCT for | poll to example.com occurs directly after a client submits an SCT for | |||
example.com, an adversary observing the auditor can trivially | example.com, an adversary observing the auditor can trivially | |||
conclude the activity of the client. | conclude the activity of the client. | |||
8.2. STH pollination | 8.2. STH pollination | |||
skipping to change at page 14, line 23 ¶ | skipping to change at page 14, line 25 ¶ | |||
contribute to the overall goal of detecting misbehaving logs by | contribute to the overall goal of detecting misbehaving logs by | |||
providing CT auditors with STHs from many vantage points, making it | providing CT auditors with STHs from many vantage points, making it | |||
possible to detect logs that are presenting inconsistent views. | possible to detect logs that are presenting inconsistent views. | |||
HTTPS servers supporting the protocol act as STH pools. HTTPS | HTTPS servers supporting the protocol act as STH pools. HTTPS | |||
clients and CT auditors in the possession of STHs can pollinate STH | clients and CT auditors in the possession of STHs can pollinate STH | |||
pools by sending STHs to them, and retrieving new STHs to send to | pools by sending STHs to them, and retrieving new STHs to send to | |||
other STH pools. CT auditors can improve the value of their auditing | other STH pools. CT auditors can improve the value of their auditing | |||
by retrieving STHs from pools. | by retrieving STHs from pools. | |||
HTPS clients send STHs to HTTPS servers by POSTing them to the well- | HTTPS clients send STHs to HTTPS servers by POSTing them to the well- | |||
known URL: | known URL: | |||
https://<domain>/.well-known/ct-gossip/v1/sth-pollination | https://<domain>/.well-known/ct-gossip/v1/sth-pollination | |||
The data sent in the POST is defined in Section 8.2.4. This data | The data sent in the POST is defined in Section 8.2.4. This data | |||
SHOULD be sent in an already established TLS session. This makes it | SHOULD be sent in an already established TLS session. This makes it | |||
hard for an attacker to disrupt STH gossiping without also disturbing | hard for an attacker to disrupt STH gossiping without also disturbing | |||
ordinary secure browsing (https://). This is discussed more in | ordinary secure browsing (https://). This is discussed more in | |||
Section 11.1.1. | Section 11.1.1. | |||
The response contains zero or more STHs in the same format, described | On a successful connection to an HTTPS server implementing STH | |||
in Section 8.2.4. | Pollination, the response code will be 200, and the response body is | |||
application/json, containing zero or more STHs in the same format, as | ||||
described in Section 8.2.4. | ||||
An HTTPS client may acquire STHs by several methods: | An HTTPS client may acquire STHs by several methods: | |||
o in replies to pollination POSTs; | o in replies to pollination POSTs; | |||
o asking logs that it recognises for the current STH, either | o asking logs that it recognizes for the current STH, either | |||
directly (v2/get-sth) or indirectly (for example over DNS) | directly (v2/get-sth) or indirectly (for example over DNS) | |||
o resolving an SCT and certificate to an STH via an inclusion proof | o resolving an SCT and certificate to an STH via an inclusion proof | |||
o resolving one STH to another via a consistency proof | o resolving one STH to another via a consistency proof | |||
HTTPS clients (that have STHs) and CT auditors SHOULD pollinate STH | HTTPS clients (that have STHs) and CT auditors SHOULD pollinate STH | |||
pools with STHs. Which STHs to send and how often pollination should | pools with STHs. Which STHs to send and how often pollination should | |||
happen is regarded as undefined policy with the exception of privacy | happen is regarded as undefined policy with the exception of privacy | |||
concerns explained below. Suggestions for the policy may be found in | concerns explained below. Suggestions for the policy can be found in | |||
Section 11.3. | Section 11.3. | |||
An HTTPS client could be tracked by giving it a unique or rare STH. | An HTTPS client could be tracked by giving it a unique or rare STH. | |||
To address this concern, we place restrictions on different | To address this concern, we place restrictions on different | |||
components of the system to ensure an STH will not be rare. | components of the system to ensure an STH will not be rare. | |||
o HTTPS clients silently ignore STHs from logs with an STH issuance | o HTTPS clients silently ignore STHs from logs with an STH issuance | |||
frequency of more than one STH per hour. Logs use the STH | frequency of more than one STH per hour. Logs use the STH | |||
Frequency Count metadata to express this ([RFC-6962-BIS-09] | Frequency Count metadata to express this ([RFC-6962-BIS-09] | |||
sections 3.6 and 5.1). | sections 3.6 and 5.1). | |||
skipping to change at page 15, line 29 ¶ | skipping to change at page 15, line 32 ¶ | |||
the past. Given a maximum STH issuance rate of one per hour, an | the past. Given a maximum STH issuance rate of one per hour, an | |||
attacker has 336 unique STHs per log for tracking. Clients MUST | attacker has 336 unique STHs per log for tracking. Clients MUST | |||
ignore STHs older than 14 days. We consider STHs within this | ignore STHs older than 14 days. We consider STHs within this | |||
validity window not to be personally identifiable data, and STHs | validity window not to be personally identifiable data, and STHs | |||
outside this window to be personally identifiable. | outside this window to be personally identifiable. | |||
When multiplied by the number of logs from which a client accepts | When multiplied by the number of logs from which a client accepts | |||
STHs, this number of unique STHs grow and the negative privacy | STHs, this number of unique STHs grow and the negative privacy | |||
implications grow with it. It's important that this is taken into | implications grow with it. It's important that this is taken into | |||
account when logs are chosen for default settings in HTTPS clients. | account when logs are chosen for default settings in HTTPS clients. | |||
This concern is discussed upon in Section 10.4.5. | This concern is discussed upon in Section 10.5.5. | |||
A log may cease operation, in which case there will soon be no STH | A log may cease operation, in which case there will soon be no STH | |||
within the validity window. Clients SHOULD perform all three methods | within the validity window. Clients SHOULD perform all three methods | |||
of gossip about a log that has ceased operation since it is possible | of gossip about a log that has ceased operation since it is possible | |||
the log was still compromised and gossip can detect that. STH | the log was still compromised and gossip can detect that. STH | |||
Pollination is the one mechanism where a client must know about a log | Pollination is the one mechanism where a client must know about a log | |||
shutdown. A client who does not know about a log shutdown MUST NOT | shutdown. A client who does not know about a log shutdown MUST NOT | |||
attempt any heuristic to detect a shutdown. Instead the client MUST | attempt any heuristic to detect a shutdown. Instead the client MUST | |||
be informed about the shutdown from a verifiable source (e.g. a | be informed about the shutdown from a verifiable source (e.g. a | |||
software update). The client SHOULD be provided the final STH issued | software update). The client SHOULD be provided the final STH issued | |||
by the log and SHOULD resolve SCTs and STHs to this final STH. If an | by the log and SHOULD resolve SCTs and STHs to this final STH. If an | |||
SCT or STH cannot be resolved to the final STH, clients should follow | SCT or STH cannot be resolved to the final STH, clients SHOULD follow | |||
the requirements and recommendations set forth in Section 11.1.2. | the requirements and recommendations set forth in Section 11.1.2. | |||
8.2.1. HTTPS Clients and Proof Fetching | 8.2.1. HTTPS Clients and Proof Fetching | |||
There are two types of proofs a client may retrieve; inclusion proofs | There are two types of proofs a client may retrieve; inclusion proofs | |||
and consistency proofs. | and consistency proofs. | |||
An HTTPS client will retrieve SCTs from an HTTPS server, and must | An HTTPS client will retrieve SCTs together with certificate chains | |||
obtain an inclusion proof to an STH in order to verify the promise | from an HTTPS server. Using the timestamp in the SCT together with | |||
made by the SCT. | the end-entity certificate and the issuer key hash, it can obtain an | |||
inclusion proof to an STH in order to verify the promise made by the | ||||
SCT. | ||||
An HTTPS client will have STHs from performing STH Pollination, and | ||||
may obtain a consistency proof to a more recent STH. | ||||
An HTTPS client may also receive an SCT bundled with an inclusion | An HTTPS client may also receive an SCT bundled with an inclusion | |||
proof to a historical STH via an unspecified future mechanism. | proof to a historical STH via an unspecified future mechanism. | |||
Because this historical STH is considered personally identifiable | Because this historical STH is considered personally identifiable | |||
information per above, the client must obtain a consistency proof to | information per above, the client needs to obtain a consistency proof | |||
a more recent STH. | to a more recent STH. | |||
A client SHOULD perform proof fetching. A client MUST NOT perform | A client SHOULD perform proof fetching. A client MUST NOT perform | |||
proof fetching for any SCTs or STHs issued by a locally added log. A | proof fetching for any SCTs or STHs issued by a locally added log. A | |||
client MAY fetch an inclusion proof for an SCT (issued by a pre- | client MAY fetch an inclusion proof for an SCT (issued by a pre- | |||
loaded log) that validates a certificate chaining to a locally added | loaded log) that validates a certificate chaining to a locally added | |||
trust anchor. | trust anchor. | |||
[ TBD: Linus doesn't like this because we're mandating behavior that | ||||
is not necessarily safe. Is it unsafe? Not sure.] | ||||
If a client requested either proof directly from a log or auditor, it | If a client requested either proof directly from a log or auditor, it | |||
would reveal the client's browsing habits to a third party. To | would reveal the client's browsing habits to a third party. To | |||
mitigate this risk, an HTTPS client MUST retrieve the proof in a | mitigate this risk, an HTTPS client MUST retrieve the proof in a | |||
manner that disguises the client. | manner that disguises the client. | |||
Depending on the client's DNS provider, DNS may provide an | Depending on the client's DNS provider, DNS may provide an | |||
appropriate intermediate layer that obfuscates the linkability | appropriate intermediate layer that obfuscates the linkability | |||
between the user of the client and the request for inclusion (while | between the user of the client and the request for inclusion (while | |||
at the same time providing a caching layer for oft-requested | at the same time providing a caching layer for oft-requested | |||
inclusion proofs.) | inclusion proofs). See [draft-ct-over-dns] for an example of how | |||
this can be done. | ||||
[ TODO: Add a reference to Google's DNS mechanism more proper than | ||||
http://www.certificate-transparency.org/august-2015-newsletter ] | ||||
Anonymity networks such as Tor also present a mechanism for a client | Anonymity networks such as Tor also present a mechanism for a client | |||
to anonymously retrieve a proof from an auditor or log. | to anonymously retrieve a proof from an auditor or log. | |||
Even when using a privacy-preserving layer between the client and the | Even when using a privacy-preserving layer between the client and the | |||
log, certain observations may be made about an anonymous client or | log, certain observations may be made about an anonymous client or | |||
general user behavior depending on how proofs are fetched. For | general user behavior depending on how proofs are fetched. For | |||
example, if a client fetched all outstanding proofs at once, a log | example, if a client fetched all outstanding proofs at once, a log | |||
would know that SCTs or STHs recieved around the same time are more | would know that SCTs or STHs received around the same time are more | |||
likely to come from a particular client. This could potentially go | likely to come from a particular client. This could potentially go | |||
so far as correlation of activity at different times to a single | so far as correlation of activity at different times to a single | |||
client. In aggregate the data could reveal what sites are commonly | client. In aggregate the data could reveal what sites are commonly | |||
visited together. HTTPS clients SHOULD use a strategy of proof | visited together. HTTPS clients SHOULD use a strategy of proof | |||
fetching that attempts to obfuscate these patterns. A suggestion of | fetching that attempts to obfuscate these patterns. A suggestion of | |||
such a policy can be found in Section 11.2. | such a policy can be found in Section 11.2. | |||
Resolving either SCTs and STHs may result in errors. These errors | Resolving either SCTs and STHs may result in errors. These errors | |||
may be routine downtime or other transient errors, or they may be | may be routine downtime or other transient errors, or they may be | |||
indicative of an attack. Clients should follow the requirements and | indicative of an attack. Clients SHOULD follow the requirements and | |||
recommendations set forth in Section 11.1.2 when handling these | recommendations set forth in Section 11.1.2 when handling these | |||
errors in order to give the CT ecosystem the greatest chance of | errors in order to give the CT ecosystem the greatest chance of | |||
detecting and responding to a compromise. | detecting and responding to a compromise. | |||
8.2.2. STH Pollination without Proof Fetching | 8.2.2. STH Pollination without Proof Fetching | |||
An HTTPS client MAY participate in STH Pollination without fetching | An HTTPS client MAY participate in STH Pollination without fetching | |||
proofs. In this situation, the client receives STHs from a server, | proofs. In this situation, the client receives STHs from a server, | |||
applies the same validation logic to them (signed by a known log, | applies the same validation logic to them (signed by a known log, | |||
within the validity window) and will later pass them to an HTTPS | within the validity window) and will later pass them to another HTTPS | |||
server. | server. | |||
When operating in this fashion, the HTTPS client is promoting gossip | When operating in this fashion, the HTTPS client is promoting gossip | |||
for Certificate Transparency, but derives no direct benefit itself. | for Certificate Transparency, but derives no direct benefit itself. | |||
In comparison, a client who resolves SCTs or historical STHs to | In comparison, a client who resolves SCTs or historical STHs to | |||
recent STHs and pollinates them is assured that if it was attacked, | recent STHs and pollinates them is assured that if it was attacked, | |||
there is a probability that the ecosystem will detect and respond to | there is a probability that the ecosystem will detect and respond to | |||
the attack (by distrusting the log). | the attack (by distrusting the log). | |||
8.2.3. Auditor Action | 8.2.3. Auditor Action | |||
CT auditors participate in STH pollination by retrieving STHs from | CT auditors participate in STH pollination by retrieving STHs from | |||
HTTPS servers. They verify that the STH is valid by checking the | HTTPS servers. They verify that the STH is valid by checking the | |||
signature, and requesting a consistency proof from the STH to the | signature, and requesting a consistency proof from the STH to the | |||
most recent STH. | most recent STH. | |||
After retrieving the consistency proof to the most recent STH, they | After retrieving the consistency proof to the most recent STH, they | |||
SHOULD pollinate this new STH among participating HTTPS Servers. In | SHOULD pollinate this new STH among participating HTTPS servers. In | |||
this way, as STHs "age out" and are no longer fresh, their "lineage" | this way, as STHs "age out" and are no longer fresh, their "lineage" | |||
continues to be tracked in the system. | continues to be tracked in the system. | |||
8.2.4. STH Pollination data format | 8.2.4. STH Pollination data format | |||
The data sent from HTTPS clients and CT auditors to HTTPS servers is | The data sent from HTTPS clients and CT auditors to HTTPS servers is | |||
a JSON object [RFC7159] with the following content: | a JSON object [RFC7159] with the following content: | |||
o sths - an array of 0 or more fresh SignedTreeHead's as defined in | o sths - an array of 0 or more fresh SignedTreeHeads as defined in | |||
[RFC-6962-BIS-09] Section 3.6.1. | [RFC-6962-BIS-09] Section 3.6.1. | |||
8.3. Trusted Auditor Stream | 8.3. Trusted Auditor Stream | |||
HTTPS clients MAY send SCTs and cert chains, as well as STHs, | HTTPS clients MAY send SCTs and cert chains, as well as STHs, | |||
directly to auditors. If sent, this data MAY include data that | directly to auditors. If sent, this data MAY include data that | |||
reflects locally added logs or trust anchors. Note that there are | reflects locally added logs or trust anchors. Note that there are | |||
privacy implications in doing so, these are outlined in | privacy implications in doing so, these are outlined in | |||
Section 10.4.1 and Section 10.4.6. | Section 10.5.1 and Section 10.5.6. | |||
The most natural trusted auditor arrangement arguably is a web | The most natural trusted auditor arrangement arguably is a web | |||
browser that is "logged in to" a provider of various internet | browser that is "logged in to" a provider of various internet | |||
services. Another equivalent arrangement is a trusted party like a | services. Another equivalent arrangement is a trusted party like a | |||
corporation to which an employee is connected through a VPN or by | corporation to which an employee is connected through a VPN or by | |||
other similar means. A third might be individuals or smaller groups | other similar means. A third might be individuals or smaller groups | |||
of people running their own services. In such a setting, retrieving | of people running their own services. In such a setting, retrieving | |||
proofs from that third party could be considered reasonable from a | proofs from that third party could be considered reasonable from a | |||
privacy perspective. The HTTPS client may also do its own auditing | privacy perspective. The HTTPS client may also do its own auditing | |||
and might additionally share SCTs and STHs with the trusted party to | and might additionally share SCTs and STHs with the trusted party to | |||
skipping to change at page 18, line 24 ¶ | skipping to change at page 18, line 29 ¶ | |||
gossip part. | gossip part. | |||
Another well established trusted party arrangement on the internet | Another well established trusted party arrangement on the internet | |||
today is the relation between internet users and their providers of | today is the relation between internet users and their providers of | |||
DNS resolver services. DNS resolvers are typically provided by the | DNS resolver services. DNS resolvers are typically provided by the | |||
internet service provider (ISP) used, which by the nature of name | internet service provider (ISP) used, which by the nature of name | |||
resolving already know a great deal about which sites their users | resolving already know a great deal about which sites their users | |||
visit. As mentioned in Section 8.2.1, in order for HTTPS clients to | visit. As mentioned in Section 8.2.1, in order for HTTPS clients to | |||
be able to retrieve proofs in a privacy preserving manner, logs could | be able to retrieve proofs in a privacy preserving manner, logs could | |||
expose a DNS interface in addition to the ordinary HTTPS interface. | expose a DNS interface in addition to the ordinary HTTPS interface. | |||
An informal writeup of such a protocol can be found at XXX. | A specification of such a protocol can be found in | |||
[draft-ct-over-dns]. | ||||
8.3.1. Trusted Auditor data format | 8.3.1. Trusted Auditor data format | |||
Trusted Auditors expose a REST API at the fixed URI: | Trusted Auditors expose a REST API at the fixed URI: | |||
https://<auditor>/ct-gossip/v1/trusted-auditor | https://<auditor>/ct-gossip/v1/trusted-auditor | |||
Submissions are made by sending an HTTPS POST request, with the body | Submissions are made by sending an HTTPS POST request, with the body | |||
of the POST in a JSON object. Upon successful receipt the Trusted | of the POST in a JSON object. Upon successful receipt the Trusted | |||
Auditor returns 200 OK. | Auditor returns 200 OK. | |||
skipping to change at page 20, line 40 ¶ | skipping to change at page 20, line 40 ¶ | |||
STH Pollination requires the cooperation of HTTPS clients, HTTPS | STH Pollination requires the cooperation of HTTPS clients, HTTPS | |||
servers, and logs. | servers, and logs. | |||
For a client to fully participate in STH Pollination, and have this | For a client to fully participate in STH Pollination, and have this | |||
mechanism detect attacks against it, the client must have a way to | mechanism detect attacks against it, the client must have a way to | |||
safely perform Proof Fetching in a privacy preserving manner. (The | safely perform Proof Fetching in a privacy preserving manner. (The | |||
client may pollinate STHs it receives without performing Proof | client may pollinate STHs it receives without performing Proof | |||
Fetching, but we do not consider this option in this section.) | Fetching, but we do not consider this option in this section.) | |||
HTTPS Servers must deploy software (although, as in the case with SCT | HTTPS servers must deploy software (although, as in the case with SCT | |||
Feedback this logic can be pre-provided) and commit some configurable | Feedback this logic can be pre-provided) and commit some configurable | |||
amount of disk space to the endeavor. | amount of disk space to the endeavor. | |||
Logs (or a third party) must provide access to clients to query | Logs (or a third party mirroring the logs) must provide access to | |||
proofs in a privacy preserving manner, most likely through DNS. | clients to query proofs in a privacy preserving manner, most likely | |||
through DNS. | ||||
Unlike SCT Feedback, the STH Pollination mechanism is not hampered if | Unlike SCT Feedback, the STH Pollination mechanism is not hampered if | |||
only a minority of HTTPS servers deploy it. However, it makes an | only a minority of HTTPS servers deploy it. However, it makes an | |||
assumption that an HTTPS client performs Proof Fetching (such as the | assumption that an HTTPS client performs Proof Fetching (such as the | |||
DNS mechanism discussed). Unfortunately, any manner that is | DNS mechanism discussed). Unfortunately, any manner that is | |||
anonymous for some (such as clients who use shared DNS services such | anonymous for some (such as clients who use shared DNS services such | |||
as a large ISP), may not be anonymous for others. | as a large ISP), may not be anonymous for others. | |||
For instance, DNS requests expose a considerable amount of sensitive | For instance, DNS requests expose a considerable amount of sensitive | |||
information (including what data is already present in the cache) in | information (including what data is already present in the cache) in | |||
plaintext over the network. For this reason, some percentage of | plaintext over the network. For this reason, some percentage of | |||
HTTPS clients may choose to not enable the Proof Fetching component | HTTPS clients may choose to not enable the Proof Fetching component | |||
of STH Pollination. (Although they can still request and send STHs | of STH Pollination. (Although they can still request and send STHs | |||
among participating HTTPS servers, even when this affords them no | among participating HTTPS servers, even when this affords them no | |||
direct benefit.) | direct benefit.) | |||
If STH Pollination was the only mechanism deployed, users that | If STH Pollination was the only mechanism deployed, users that | |||
disable it would be able to be attacked without risk of detection. | disable it would be able to be attacked without risk of detection. | |||
If STH Pollination was not deployed, HTTPS Clients visiting HTTPS | If STH Pollination was not deployed, HTTPS clients visiting HTTPS | |||
Servers who did not deploy SCT Feedback could be attacked without | Servers who did not deploy SCT Feedback could be attacked without | |||
risk of detection. | risk of detection. | |||
9.3. Trusted Auditor Relationship | 9.3. Trusted Auditor Relationship | |||
The Trusted Auditor Relationship is expected to be the rarest gossip | The Trusted Auditor Relationship is expected to be the rarest gossip | |||
mechanism, as an HTTPS Client is providing an unadulterated report of | mechanism, as an HTTPS client is providing an unadulterated report of | |||
its browsing history to a third party. While there are valid and | its browsing history to a third party. While there are valid and | |||
common reasons for doing so, there is no appropriate way to enter | common reasons for doing so, there is no appropriate way to enter | |||
into this relationship without retrieving informed consent from the | into this relationship without retrieving informed consent from the | |||
user. | user. | |||
However, the Trusted Auditor Relationship mechanism still provides | However, the Trusted Auditor Relationship mechanism still provides | |||
value to a class of HTTPS Clients. For example, web crawlers have no | value to a class of HTTPS clients. For example, web crawlers have no | |||
concept of a "user" and no expectation of privacy. Organizations | concept of a "user" and no expectation of privacy. Organizations | |||
already performing network auditing for anomalies or attacks can run | already performing network auditing for anomalies or attacks can run | |||
their own Trusted Auditor for the same purpose with marginal increase | their own Trusted Auditor for the same purpose with marginal increase | |||
in privacy concerns. | in privacy concerns. | |||
The ability to change one's Trusted Auditor is a form of Trust | The ability to change one's Trusted Auditor is a form of Trust | |||
Agility that allows a user to choose who to trust, and be able to | Agility that allows a user to choose who to trust, and be able to | |||
revise that decision later without consequence. A Trusted Auditor | revise that decision later without consequence. A Trusted Auditor | |||
connection can be made more confidential than DNS (through the use of | connection can be made more confidential than DNS (through the use of | |||
TLS), and can even be made (somewhat) anonymous through the use of | TLS), and can even be made (somewhat) anonymous through the use of | |||
skipping to change at page 22, line 11 ¶ | skipping to change at page 22, line 16 ¶ | |||
organizations would build it themselves for their own needs. By | organizations would build it themselves for their own needs. By | |||
standardizing it, users who wish to opt-in (for instance those | standardizing it, users who wish to opt-in (for instance those | |||
unwilling to participate fully in STH Pollination) can have an | unwilling to participate fully in STH Pollination) can have an | |||
interoperable standard they can use to choose and change their | interoperable standard they can use to choose and change their | |||
trusted auditor. | trusted auditor. | |||
9.4. Interaction | 9.4. Interaction | |||
The interactions of the mechanisms is thus outlined: | The interactions of the mechanisms is thus outlined: | |||
HTTPS Clients can be attacked without risk of detection if they do | HTTPS clients can be attacked without risk of detection if they do | |||
not participate in any of the three mechanisms. | not participate in any of the three mechanisms. | |||
HTTPS Clients are afforded the greatest chance of detecting an attack | HTTPS clients are afforded the greatest chance of detecting an attack | |||
when they either participate in both SCT Feedback and STH Pollination | when they either participate in both SCT Feedback and STH Pollination | |||
with Proof Fetching or if they have a Trusted Auditor relationship. | with Proof Fetching or if they have a Trusted Auditor relationship. | |||
(Participating in SCT Feedback is required to prevent a malicious log | (Participating in SCT Feedback is required to prevent a malicious log | |||
from refusing to ever resolve an SCT to an STH, as put forward in | from refusing to ever resolve an SCT to an STH, as put forward in | |||
Section 10.1). Additionally, participating in SCT Feedback enables | Section 10.1). Additionally, participating in SCT Feedback enables | |||
an HTTPS Client to assist in detecting the exact target of an attack. | an HTTPS client to assist in detecting the exact target of an attack. | |||
HTTPS Servers that omit SCT Feedback enable malicious logs to carry | HTTPS servers that omit SCT Feedback enable malicious logs to carry | |||
out attacks without risk of detection. If these servers are targeted | out attacks without risk of detection. If these servers are targeted | |||
specifically, even if the attack is detected, without SCT Feedback | specifically, even if the attack is detected, without SCT Feedback | |||
they may never learn that they were specifically targeted. HTTPS | they may never learn that they were specifically targeted. HTTPS | |||
servers without SCT Feedback do gain some measure of herd immunity, | servers without SCT Feedback do gain some measure of herd immunity, | |||
but only because their clients participate in STH Pollination (with | but only because their clients participate in STH Pollination (with | |||
Proof Fetching) or have a Trusted Auditor Relationship. | Proof Fetching) or have a Trusted Auditor Relationship. | |||
When HTTPS Servers omit SCT feedback, it allows their users to be | When HTTPS servers omit SCT feedback, it allows their users to be | |||
attacked without detection by a malicious log; the vulnerable users | attacked without detection by a malicious log; the vulnerable users | |||
are those who do not have a Trusted Auditor relationship. | are those who do not have a Trusted Auditor relationship. | |||
10. Security considerations | 10. Security considerations | |||
10.1. Attacks by actively malicious logs | 10.1. Attacks by actively malicious logs | |||
One of the most powerful attacks possible in the CT ecosystem is a | One of the most powerful attacks possible in the CT ecosystem is a | |||
trusted log that has actively decided to be malicious. It can carry | trusted log that has actively decided to be malicious. It can carry | |||
out an attack in two ways: | out an attack in two ways: | |||
skipping to change at page 23, line 15 ¶ | skipping to change at page 23, line 19 ¶ | |||
(Alternately, it can include it in a branch of the tree and issue an | (Alternately, it can include it in a branch of the tree and issue an | |||
STH, but then abandon that branch.) Whenever someone requests an | STH, but then abandon that branch.) Whenever someone requests an | |||
inclusion proof for that SCT (or a consistency proof from that STH), | inclusion proof for that SCT (or a consistency proof from that STH), | |||
the log would respond with an error, and a client may simply regard | the log would respond with an error, and a client may simply regard | |||
the response as a transient error. This attack can be detected using | the response as a transient error. This attack can be detected using | |||
SCT Feedback, or an Auditor of Last Resort, as presented in | SCT Feedback, or an Auditor of Last Resort, as presented in | |||
Section 11.1.2. | Section 11.1.2. | |||
10.2. Dual-CA Compromise | 10.2. Dual-CA Compromise | |||
XXX describes an attack possible by an adversary who compromises two | [dual-ca-compromise-attack] describes an attack possible by an | |||
Certificate Authorites and a Log. This attack is difficult to defend | adversary who compromises two Certificate Authorities and a Log. This | |||
against in the CT ecosystem, and XXX describes a few approaches to | attack is difficult to defend against in the CT ecosystem, and | |||
doing so. We note that Gossip is not intended to defend against this | [dual-ca-compromise-attack] describes a few approaches to doing so. | |||
attack, but can in certain modes. | We note that Gossip is not intended to defend against this attack, | |||
but can in certain modes. | ||||
Defending against the Dual-CA Compromise attack requires SCT | Defending against the Dual-CA Compromise attack requires SCT | |||
Feedback, and explicitly requires the server to save full certificate | Feedback, and explicitly requires the server to save full certificate | |||
chains (described in Section 8.1.3 as the 'complex' configuration.) | chains (described in Section 8.1.3 as the 'complex' configuration.) | |||
After CT auditors receive the full certificate chains from servers, | After CT auditors receive the full certificate chains from servers, | |||
they must compare the chain built by clients to the chain supplied by | they MAY compare the chain built by clients to the chain supplied by | |||
the log. If the chains differ significantly, the auditor can raise a | the log. If the chains differ significantly, the auditor SHOULD | |||
concern. | raise a concern. A method of determining if chains differ | |||
significantly is by asserting that one chain is not a subset of the | ||||
other and that the roots of the chains are different. | ||||
[ What does 'differ significantly' mean? We should provide guidance. | [Note: Justification for this algorithm: | |||
I _think_ the correct algorithm to raise a concern is: | ||||
If one chain is not a subset of the other AND If the root | Cross-Signatures could result in a different org being treated as the | |||
certificates of the chains are different THEN It's suspicious. | 'root', but in this case, one chain would be a subset of the other. | |||
Justification: - Cross-Signatures could result in a different org | Intermediate swapping (e.g. different signature algorithms) could | |||
being treated as the 'root', but in this case, one chain would be a | result in different chains, but the root would be the same. | |||
subset of the other. - Intermediate swapping (e.g. different | ||||
signature algorithms) could result in different chains, but the root | ||||
would be the same. | ||||
(Hitting both those cases at once would cause a false positive | (Hitting both those cases at once would cause a false positive | |||
though.) | though, but this would likely be rare.) | |||
What did I miss? ] | Are there other cases that could occur? (Left for the purposes of | |||
reading during pre-Last Call, to be removed by Editor)] | ||||
10.3. Censorship/Blocking considerations | 10.3. Censorship/Blocking considerations | |||
We assume a network attacker who is able to fully control the | We assume a network attacker who is able to fully control the | |||
client's internet connection for some period of time, including | client's internet connection for some period of time, including | |||
selectively blocking requests to certain hosts and truncating TLS | selectively blocking requests to certain hosts and truncating TLS | |||
connections based on information observed or guessed about client | connections based on information observed or guessed about client | |||
behavior. In order to successfully detect log misbehavior, the | behavior. In order to successfully detect log misbehavior, the | |||
gossip mechanisms must still work even in these conditions. | gossip mechanisms must still work even in these conditions. | |||
skipping to change at page 24, line 40 ¶ | skipping to change at page 24, line 47 ¶ | |||
was not blocked. In this situation, the party can factor this into | was not blocked. In this situation, the party can factor this into | |||
strategies suggested in Section 11.3 and in Section 11.1.2. | strategies suggested in Section 11.3 and in Section 11.1.2. | |||
The connections that allow positive affirmation are 1, 2, 4, 5, and | The connections that allow positive affirmation are 1, 2, 4, 5, and | |||
7. | 7. | |||
More insidious is blocking the connections that do not allow positive | More insidious is blocking the connections that do not allow positive | |||
confirmation: 3 and 6. An attacker may truncate or drop a response | confirmation: 3 and 6. An attacker may truncate or drop a response | |||
from a server to a client, such that the server believes it has | from a server to a client, such that the server believes it has | |||
shared data with the recipient, when it has not. However, in both | shared data with the recipient, when it has not. However, in both | |||
scenatios (3 and 6), the server cannot distinguish the client as a | scenarios (3 and 6), the server cannot distinguish the client as a | |||
cooperating member of the CT ecosystem or as an attacker performing a | cooperating member of the CT ecosystem or as an attacker performing a | |||
sybil attack, aiming to flush the server's data store. Therefore the | Sybil attack, aiming to flush the server's data store. Therefore the | |||
fact that these connections can be undetectably blocked does not | fact that these connections can be undetectably blocked does not | |||
actually alter the threat model of servers responding to these | actually alter the threat model of servers responding to these | |||
requests. The choice of algorithm to release data is crucial to | requests. The choice of algorithm to release data is crucial to | |||
protect against these attacks; strategies are suggested in | protect against these attacks; strategies are suggested in | |||
Section 11.3. | Section 11.3. | |||
Handling censorship and network blocking (which is indistinguishable | Handling censorship and network blocking (which is indistinguishable | |||
from network error) is relegated to the implementation policy chosen | from network error) is relegated to the implementation policy chosen | |||
by clients. Suggestions for client behavior are specified in | by clients. Suggestions for client behavior are specified in | |||
Section 11.1. | Section 11.1. | |||
10.4. Privacy considerations | 10.4. Flushing Attacks | |||
CT Gossip deals with HTTPS Clients which are trying to share | A flushing attack is an attempt by an adversary to flush a particular | |||
piece of data from a pool. In the CT Gossip ecosystem, an attacker | ||||
may have performed an attack and left evidence of a compromised log | ||||
on a client or server. They would be interested in flushing that | ||||
data, i.e. tricking the target into gossiping or pollinating the | ||||
incriminating evidence with only attacker-controlled clients or | ||||
servers with the hope they trick the target into deleting it. | ||||
Flushing attacks may be defended against differently depending on the | ||||
entity (HTTPS client or HTTPS server) and record (STHs or SCTs with | ||||
Certificate Chains). | ||||
10.4.1. STHs | ||||
For both HTTPS clients and HTTPS servers, STHs within the validity | ||||
window SHOULD NOT be deleted. An attacker cannot flush an item from | ||||
the cache if it is never removed so flushing attacks are completely | ||||
mitigated. | ||||
The required disk space for all STHs within the validity window is | ||||
336 STHs per log that is trusted. If 20 logs are trusted, and each | ||||
STH takes 1 Kilobytes, this is 6.56 Megabytes. | ||||
Note that it is important that implementors do not calculate the | ||||
exact size of cache expected - if an attack does occur, a small | ||||
number of additional STHs will enter into the cache. These STHs will | ||||
be in addition to the expected set, and will be evidence of the | ||||
attack. | ||||
If an HTTPS client or HTTPS server is operating in a constrained | ||||
environment and cannot devote enough storage space to hold all STHs | ||||
within the validity window it is recommended to use the below | ||||
Deletion Algorithm Section 11.3.2 to make it more difficult for the | ||||
attacker to perform a flushing attack. | ||||
10.4.2. SCTs & Certificate Chains on HTTPS Servers | ||||
An HTTPS server will only accept SCTs and Certificate Chains for | ||||
domains it is authoritative for. Therefore the storage space needed | ||||
is bound by the number of logs it accepts, multiplied by the number | ||||
of domains it is authoritative for, multiplied by the number of | ||||
certificates issued for those domains. | ||||
Imagine a server authoritative for 10,000 domains, and each domain | ||||
has 3 certificate chains, and 10 SCTs. A certificate chain is 5 | ||||
Kilobytes in size and an SCT 1 Kilobyte. This yields 732 Megabytes. | ||||
This data can be large, but it is calculable. Web properties with | ||||
more certificates and domains are more likely to be able to handle | ||||
the increased storage need, while small web properties will not seen | ||||
an undue burden. Therefore HTTPS servers SHOULD NOT delete SCTs or | ||||
Certificate Chains. This completely mitigates flushing attacks. | ||||
Again, note that it is important that implementors do not calculate | ||||
the exact size of cache expected - if an attack does occur, the new | ||||
SCT(s) and Certificate Chain(s) will enter into the cache. This data | ||||
will be in addition to the expected set, and will be evidence of the | ||||
attack. | ||||
If an HTTPS server is operating in a constrained environment and | ||||
cannot devote enough storage space to hold all SCTs and Certificate | ||||
Chains it is authoritative for it is recommended to configure the SCT | ||||
Feedback mechanism to allow only certain certificates that are known | ||||
to be valid. These chains and SCTs can then be discarded without | ||||
being stored or subsequently provided to any clients or auditors. If | ||||
the allowlist is not sufficient, the below Deletion Algorithm | ||||
Section 11.3.2 is recommended to make it more difficult for the | ||||
attacker to perform a flushing attack. | ||||
10.4.3. SCTs & Certificate Chains on HTTPS Clients | ||||
HTTPS clients will accumulate SCTs and Certificate Chains without | ||||
bound. It is expected they will choose a particular cache size and | ||||
delete entries when the cache size meets its limit. This does not | ||||
mitigate flushing attacks, and such an attack is documented in | ||||
[gossip-mixing]. | ||||
The below Deletion Algorithm Section 11.3.2 is recommended to make it | ||||
more difficult for the attacker to perform a flushing attack. | ||||
10.5. Privacy considerations | ||||
CT Gossip deals with HTTPS clients which are trying to share | ||||
indicators that correspond to their browsing history. The most | indicators that correspond to their browsing history. The most | |||
sensitive relationships in the CT ecosystem are the relationships | sensitive relationships in the CT ecosystem are the relationships | |||
between HTTPS clients and HTTPS servers. Client-server relationships | between HTTPS clients and HTTPS servers. Client-server relationships | |||
can be aggregated into a network graph with potentially serious | can be aggregated into a network graph with potentially serious | |||
implications for correlative de-anonymisation of clients and | implications for correlative de-anonymization of clients and | |||
relationship-mapping or clustering of servers or of clients. | relationship-mapping or clustering of servers or of clients. | |||
There are, however, certain clients that do not require privacy | There are, however, certain clients that do not require privacy | |||
protection. Examples of these clients are web crawlers or robots. | protection. Examples of these clients are web crawlers or robots. | |||
But even in this case, the method by which these clients crawl the | But even in this case, the method by which these clients crawl the | |||
web may in fact be considered sensitive information. In general, it | web may in fact be considered sensitive information. In general, it | |||
is better to err on the side of safety, and not assume a client is | is better to err on the side of safety, and not assume a client is | |||
okay with giving up its privacy. | okay with giving up its privacy. | |||
10.4.1. Privacy and SCTs | 10.5.1. Privacy and SCTs | |||
An SCT contains information that links it to a particular web site. | An SCT contains information that links it to a particular web site. | |||
Because the client-server relationship is sensitive, gossip between | Because the client-server relationship is sensitive, gossip between | |||
clients and servers about unrelated SCTs is risky. Therefore, a | clients and servers about unrelated SCTs is risky. Therefore, a | |||
client with an SCT for a given server should transmit that | client with an SCT for a given server SHOULD NOT transmit that | |||
information in only two channels: to the server associated with the | information in any other than the following two channels: to the | |||
SCT itself; and to a Trusted Auditor, if one exists. | server associated with the SCT itself; or to a Trusted Auditor, if | |||
one exists. | ||||
10.4.2. Privacy in SCT Feedback | 10.5.2. Privacy in SCT Feedback | |||
SCTs introduce yet another mechanism for HTTPS servers to store state | SCTs introduce yet another mechanism for HTTPS servers to store state | |||
on an HTTPS client, and potentially track users. HTTPS clients which | on an HTTPS client, and potentially track users. HTTPS clients which | |||
allow users to clear history or cookies associated with an origin | allow users to clear history or cookies associated with an origin | |||
MUST clear stored SCTs and certificate chains associated with the | MUST clear stored SCTs and certificate chains associated with the | |||
origin as well. | origin as well. | |||
Auditors should treat all SCTs as sensitive data. SCTs received | Auditors should treat all SCTs as sensitive data. SCTs received | |||
directly from an HTTPS client are especially sensitive, because the | directly from an HTTPS client are especially sensitive, because the | |||
auditor is a trusted by the client to not reveal their associations | auditor is a trusted by the client to not reveal their associations | |||
skipping to change at page 26, line 10 ¶ | skipping to change at page 28, line 8 ¶ | |||
Section 11.3. | Section 11.3. | |||
There is a possible fingerprinting attack where a log issues a unique | There is a possible fingerprinting attack where a log issues a unique | |||
SCT for targeted log client(s). A colluding log and HTTPS server | SCT for targeted log client(s). A colluding log and HTTPS server | |||
operator could therefore be a threat to the privacy of an HTTPS | operator could therefore be a threat to the privacy of an HTTPS | |||
client. Given all the other opportunities for HTTPS servers to | client. Given all the other opportunities for HTTPS servers to | |||
fingerprint clients - TLS session tickets, HPKP and HSTS headers, | fingerprint clients - TLS session tickets, HPKP and HSTS headers, | |||
HTTP Cookies, etc. - this is considered acceptable. | HTTP Cookies, etc. - this is considered acceptable. | |||
The fingerprinting attack described above would be mitigated by a | The fingerprinting attack described above would be mitigated by a | |||
requirement that logs MUST use a deterministic signature scheme when | requirement that logs must use a deterministic signature scheme when | |||
signing SCTs ([RFC-6962-BIS-09] Section 2.1.4). A log signing using | signing SCTs ([RFC-6962-BIS-09] Section 2.1.4). A log signing using | |||
RSA is not required to use a deterministic signature scheme. | RSA is not required to use a deterministic signature scheme. | |||
Since logs are allowed to issue a new SCT for a certificate already | Since logs are allowed to issue a new SCT for a certificate already | |||
present in the log, mandating deterministic signatures does not stop | present in the log, mandating deterministic signatures does not stop | |||
this fingerprinting attack altogether. It does make the attack | this fingerprinting attack altogether. It does make the attack | |||
harder to pull off without being detected though. | harder to pull off without being detected though. | |||
There is another similar fingerprinting attack where an HTTPS server | There is another similar fingerprinting attack where an HTTPS server | |||
tracks a client by using a unqiue certificate or a variation of cert | tracks a client by using a unique certificate or a variation of cert | |||
chains. The risk for this attack is accepted on the same grounds as | chains. The risk for this attack is accepted on the same grounds as | |||
the unique SCT attack described above. [XXX any mitigations possible | the unique SCT attack described above. | |||
here?] | ||||
10.4.3. Privacy for HTTPS clients performing STH Proof Fetching | 10.5.3. Privacy for HTTPS clients performing STH Proof Fetching | |||
An HTTPS client performing Proof Fetching should only request proofs | An HTTPS client performing Proof Fetching SHOULD NOT request proofs | |||
from a CT log that it accepts SCTs from. An HTTPS client MAY [TBD | from a CT log that it doesn't accept SCTs from. An HTTPS client | |||
SHOULD?] regularly request an STH from all logs it is willing to | SHOULD regularly request an STH from all logs it is willing to | |||
accept, even if it has seen no SCTs from that log. | accept, even if it has seen no SCTs from that log. | |||
[ TBD how regularly? This has operational implications for log | The time between two polls for new STH's SHOULD NOT be significantly | |||
operators ] | shorter than the MMD of the polled log divided by its STH Frequency | |||
Count ([RFC-6962-BIS-09] section 5.1). | ||||
The actual mechanism by which Proof Fetching is done carries | The actual mechanism by which Proof Fetching is done carries | |||
considerable privacy concerns. Although out of scope for the | considerable privacy concerns. Although out of scope for the | |||
document, DNS is a mechanism currently discussed. DNS exposes data | document, DNS is a mechanism currently discussed. DNS exposes data | |||
in plaintext over the network (including what sites the user is | in plaintext over the network (including what sites the user is | |||
visiting and what sites they have previously visited) an may not be | visiting and what sites they have previously visited) and may not be | |||
suitable for some. | suitable for some. | |||
10.4.4. Privacy in STH Pollination | 10.5.4. Privacy in STH Pollination | |||
An STH linked to an HTTPS client may indicate the following about | An STH linked to an HTTPS client may indicate the following about | |||
that client: | that client: | |||
o that the client gossips; | o that the client gossips; | |||
o that the client has been using CT at least until the time that the | o that the client has been using CT at least until the time that the | |||
timestamp and the tree size indicate; | timestamp and the tree size indicate; | |||
o that the client is talking, possibly indirectly, to the log | o that the client is talking, possibly indirectly, to the log | |||
indicated by the tree hash; | indicated by the tree hash; | |||
o which software and software version is being used. | o which software and software version is being used. | |||
There is a possible fingerprinting attack where a log issues a unique | There is a possible fingerprinting attack where a log issues a unique | |||
STH for a targeted HTTPS client. This is similar to the | STH for a targeted HTTPS client. This is similar to the | |||
fingerprinting attack described in Section 10.4.2, but can operate | fingerprinting attack described in Section 10.5.2, but can operate | |||
cross-origin. If a log (or HTTPS Server cooperating with a log) | cross-origin. If a log (or HTTPS server cooperating with a log) | |||
provides a unique STH to a client, the targeted client will be the | provides a unique STH to a client, the targeted client will be the | |||
only client pollinating that STH cross-origin. | only client pollinating that STH cross-origin. | |||
It is mitigated partially because the log is limited in the number of | It is mitigated partially because the log is limited in the number of | |||
STHs it can issue. It must 'save' one of its STHs each MMD to | STHs it can issue. It must 'save' one of its STHs each MMD to | |||
perform the attack. | perform the attack. | |||
10.4.5. Privacy in STH Interaction | 10.5.5. Privacy in STH Interaction | |||
An HTTPS client may pollinate any STH within the last 14 days. An | An HTTPS client may pollinate any STH within the last 14 days. An | |||
HTTPS Client may also pollinate an STH for any log that it knows | HTTPS client may also pollinate an STH for any log that it knows | |||
about. When a client pollinates STHs to a server, it will release | about. When a client pollinates STHs to a server, it will release | |||
more than one STH at a time. It is unclear if a server may 'prime' a | more than one STH at a time. It is unclear if a server may 'prime' a | |||
client and be able to reliably detect the client at a later time. | client and be able to reliably detect the client at a later time. | |||
It's clear that a single site can track a user any way they wish, but | It's clear that a single site can track a user any way they wish, but | |||
this attack works cross-origin and is therefore more concerning. Two | this attack works cross-origin and is therefore more concerning. Two | |||
independent sites A and B want to collaborate to track a user cross- | independent sites A and B want to collaborate to track a user cross- | |||
origin. A feeds a client Carol some N specific STHs from the M logs | origin. A feeds a client Carol some N specific STHs from the M logs | |||
Carol trusts, chosen to be older and less common, but still in the | Carol trusts, chosen to be older and less common, but still in the | |||
validity window. Carol visits B and chooses to release some of the | validity window. Carol visits B and chooses to release some of the | |||
skipping to change at page 28, line 5 ¶ | skipping to change at page 29, line 46 ¶ | |||
which of those STHs to send to B, it should be possible to calculate | which of those STHs to send to B, it should be possible to calculate | |||
statistics about how unique Carol looks when talking to B and how | statistics about how unique Carol looks when talking to B and how | |||
useful/accurate such a tracking mechanism is. | useful/accurate such a tracking mechanism is. | |||
Building such a model is likely impossible without some real world | Building such a model is likely impossible without some real world | |||
data, and requires a given implementation of a policy. To combat | data, and requires a given implementation of a policy. To combat | |||
this attack, suggestions are provided in Section 11.3 to attempt to | this attack, suggestions are provided in Section 11.3 to attempt to | |||
minimize it, but follow-up testing with real world deployment to | minimize it, but follow-up testing with real world deployment to | |||
improve the policy will be required. | improve the policy will be required. | |||
10.4.6. Trusted Auditors for HTTPS Clients | 10.5.6. Trusted Auditors for HTTPS Clients | |||
Some HTTPS clients may choose to use a trusted auditor. This trust | Some HTTPS clients may choose to use a trusted auditor. This trust | |||
relationship exposes a large amount of information about the client | relationship exposes a large amount of information about the client | |||
to the auditor. In particular, it will identify the web sites that | to the auditor. In particular, it will identify the web sites that | |||
the client has visited to the auditor. Some clients may already | the client has visited to the auditor. Some clients may already | |||
share this information to a third party, for example, when using a | share this information to a third party, for example, when using a | |||
server to synchronize browser history across devices in a server- | server to synchronize browser history across devices in a server- | |||
visible way, or when doing DNS lookups through a trusted DNS | visible way, or when doing DNS lookups through a trusted DNS | |||
resolver. For clients with such a relationship already established, | resolver. For clients with such a relationship already established, | |||
sending SCTs to a trusted auditor run by the same organization does | sending SCTs to a trusted auditor run by the same organization does | |||
skipping to change at page 28, line 30 ¶ | skipping to change at page 30, line 22 ¶ | |||
identities with their SCTs may wish to use an anonymizing network | identities with their SCTs may wish to use an anonymizing network | |||
like Tor to submit SCT Feedback to the auditor. Auditors SHOULD | like Tor to submit SCT Feedback to the auditor. Auditors SHOULD | |||
accept SCT Feedback that arrives over such anonymizing networks. | accept SCT Feedback that arrives over such anonymizing networks. | |||
Clients sending feedback to an auditor may prefer to reduce the | Clients sending feedback to an auditor may prefer to reduce the | |||
temporal granularity of the history exposure to the auditor by | temporal granularity of the history exposure to the auditor by | |||
caching and delaying their SCT Feedback reports. This is elaborated | caching and delaying their SCT Feedback reports. This is elaborated | |||
upon in Section 11.3. This strategy is only as effective as the | upon in Section 11.3. This strategy is only as effective as the | |||
granularity of the timestamps embedded in the SCTs and STHs. | granularity of the timestamps embedded in the SCTs and STHs. | |||
10.4.7. HTTPS Clients as Auditors | 10.5.7. HTTPS Clients as Auditors | |||
Some HTTPS Clients may choose to act as CT auditors themselves. A | Some HTTPS clients may choose to act as CT auditors themselves. A | |||
Client taking on this role needs to consider the following: | Client taking on this role needs to consider the following: | |||
o an Auditing HTTPS Client potentially exposes its history to the | o an Auditing HTTPS client potentially exposes its history to the | |||
logs that they query. Querying the log through a cache or a proxy | logs that they query. Querying the log through a cache or a proxy | |||
with many other users may avoid this exposure, but may expose | with many other users may avoid this exposure, but may expose | |||
information to the cache or proxy, in the same way that a non- | information to the cache or proxy, in the same way that a non- | |||
Auditing HTTPS Client exposes information to a Trusted Auditor. | Auditing HTTPS Client exposes information to a Trusted Auditor. | |||
o an effective CT auditor needs a strategy about what to do in the | o an effective CT auditor needs a strategy about what to do in the | |||
event that it discovers misbehavior from a log. Misbehavior from | event that it discovers misbehavior from a log. Misbehavior from | |||
a log involves the log being unable to provide either (a) a | a log involves the log being unable to provide either (a) a | |||
consistency proof between two valid STHs or (b) an inclusion proof | consistency proof between two valid STHs or (b) an inclusion proof | |||
for a certificate to an STH any time after the log's MMD has | for a certificate to an STH any time after the log's MMD has | |||
elapsed from the issuance of the SCT. The log's inability to | elapsed from the issuance of the SCT. The log's inability to | |||
provide either proof will not be externally cryptographically- | provide either proof will not be externally cryptographically- | |||
verifiable, as it may be indistinguishable from a network error. | verifiable, as it may be indistinguishable from a network error. | |||
11. Policy Recommendations | 11. Policy Recommendations | |||
This section is intended as suggestions to implementors of HTTPS | This section is intended as suggestions to implementors of HTTPS | |||
Clients, HTTPS Servers, and CT auditors. It is not a requirement for | Clients, HTTPS servers, and CT auditors. It is not a requirement for | |||
technique of implementation, so long as privacy considerations | technique of implementation, so long as privacy considerations | |||
established above are obeyed. | established above are obeyed. | |||
11.1. Blocking Recommendations | 11.1. Blocking Recommendations | |||
11.1.1. Frustrating blocking | 11.1.1. Frustrating blocking | |||
When making gossip connections to HTTPS Servers or Trusted Auditors, | When making gossip connections to HTTPS servers or Trusted Auditors, | |||
it is desirable to minimize the plaintext metadata in the connection | it is desirable to minimize the plaintext metadata in the connection | |||
that can be used to identify the connection as a gossip connection | that can be used to identify the connection as a gossip connection | |||
and therefore be of interest to block. Additionally, introducing | and therefore be of interest to block. Additionally, introducing | |||
some randomness into client behavior may be important. We assume | some randomness into client behavior may be important. We assume | |||
that the adversary is able to inspect the behavior of the HTTPS | that the adversary is able to inspect the behavior of the HTTPS | |||
client and understand how it makes gossip connections. | client and understand how it makes gossip connections. | |||
As an example, if a client, after establishing a TLS connection (and | As an example, if a client, after establishing a TLS connection (and | |||
receiving an SCT, but not making its own HTTP request yet), | receiving an SCT, but not making its own HTTP request yet), | |||
immediately opens a second TLS connection for the purpose of gossip, | immediately opens a second TLS connection for the purpose of gossip, | |||
the adversary can reliably block this second connection to block | the adversary can reliably block this second connection to block | |||
gossip without affecting normal browsing. For this reason it is | gossip without affecting normal browsing. For this reason it is | |||
recommended to run the gossip protocols over an existing connection | recommended to run the gossip protocols over an existing connection | |||
to the server, making use of connection multiplexing such as HTTP | to the server, making use of connection multiplexing such as HTTP | |||
Keep-Alives or SPDY. | Keep-Alive or SPDY. | |||
Truncation is also a concern. If a client always establishes a TLS | Truncation is also a concern. If a client always establishes a TLS | |||
connection, makes a request, receives a response, and then always | connection, makes a request, receives a response, and then always | |||
attempts a gossip communication immediately following the first | attempts a gossip communication immediately following the first | |||
response, truncation will allow an attacker to block gossip reliably. | response, truncation will allow an attacker to block gossip reliably. | |||
For these reasons, we recommend that, if at all possible, clients | For these reasons, we recommend that, if at all possible, clients | |||
SHOULD send gossip data in an already established TLS session. This | SHOULD send gossip data in an already established TLS session. This | |||
can be done through the use of HTTP Pipelining, SPDY, or HTTP/2. | can be done through the use of HTTP Pipelining, SPDY, or HTTP/2. | |||
11.1.2. Responding to possible blocking | 11.1.2. Responding to possible blocking | |||
In some cirsumstances a client may have a piece of data that they | In some circumstances a client may have a piece of data that they | |||
have attempted to share (via SCT Feedback or STH Pollination), but | have attempted to share (via SCT Feedback or STH Pollination), but | |||
have been unable to do so: with every attempt they recieve an error. | have been unable to do so: with every attempt they receive an error. | |||
These situations are: | These situations are: | |||
1. The client has an SCT and a certificate, and attempts to retrieve | 1. The client has an SCT and a certificate, and attempts to retrieve | |||
an inclusion proof - but recieves an error on every attempt. | an inclusion proof - but receives an error on every attempt. | |||
2. The client has an STH, and attempts to resolve it to a newer STH | 2. The client has an STH, and attempts to resolve it to a newer STH | |||
via a consistency proof - but recieves an error on every attempt. | via a consistency proof - but receives an error on every attempt. | |||
3. The client has attempted to share an SCT and constructed | 3. The client has attempted to share an SCT and constructed | |||
certificate via SCT Feedback - but recieves an error on every | certificate via SCT Feedback - but receives an error on every | |||
attempt. | attempt. | |||
4. The client has attempted to share an STH via STH Pollination - | 4. The client has attempted to share an STH via STH Pollination - | |||
but recieves an error on every attempt. | but receives an error on every attempt. | |||
5. The client has attempted to share a specific piece of data with a | 5. The client has attempted to share a specific piece of data with a | |||
Trusted Auditor - but recieves an error on every attempt. | Trusted Auditor - but receives an error on every attempt. | |||
In the case of 1 or 2, it is conceivable that the reason for the | In the case of 1 or 2, it is conceivable that the reason for the | |||
errors is that the log acted improperly, either through malicious | errors is that the log acted improperly, either through malicious | |||
actions or compromise. A proof may not be able to be fetched because | actions or compromise. A proof may not be able to be fetched because | |||
it does not exist (and only errors or timeouts occur). One such | it does not exist (and only errors or timeouts occur). One such | |||
situation may arise because of an actively malicious log, as | situation may arise because of an actively malicious log, as | |||
presented in Section 10.1. This data is especially important to | presented in Section 10.1. This data is especially important to | |||
share with the broader internet to detect this situation. | share with the broader internet to detect this situation. | |||
If an SCT has attempted to be resolved to an STH via an inclusion | If an SCT has attempted to be resolved to an STH via an inclusion | |||
proof multiple times, and each time has failed, a client SHOULD make | proof multiple times, and each time has failed, this SCT might very | |||
every effort to send this SCT via SCT Feedback. However the client | well be a compromising proof of an attack. However the client MUST | |||
MUST NOT share the data with any other third party (excepting a | NOT share the data with any other third party (excepting a Trusted | |||
Trusted Auditor should one exist). | Auditor should one exist). | |||
If an STH has attempted to be resolved to a newer STH via a | If an STH has attempted to be resolved to a newer STH via a | |||
consistency proof multiple times, and each time has failed, a client | consistency proof multiple times, and each time has failed, a client | |||
MAY share the STH with an "Auditor of Last Resort" even if the STH in | MAY share the STH with an "Auditor of Last Resort" even if the STH in | |||
question is no longer within the validity window. This auditor may | question is no longer within the validity window. This auditor may | |||
be pre-configured in the client, but the client SHOULD permit a user | be pre-configured in the client, but the client SHOULD permit a user | |||
to disable the functionality or change whom data is sent to. The | to disable the functionality or change whom data is sent to. The | |||
Auditor of Last Resort itself represents a point of failure, so if | Auditor of Last Resort itself represents a point of failure and | |||
implemented, it should connect using public key pinning and not | privacy concerns, so if implemented, it SHOULD connect using public | |||
considered an item delivered until it recieves a confirmation. | key pinning and not consider an item delivered until it receives a | |||
confirmation. | ||||
In the cases 3, 4, and 5, we assume that the webserver(s) or trusted | In the cases 3, 4, and 5, we assume that the webserver(s) or trusted | |||
auditor in question is either experiencing an operational failure, or | auditor in question is either experiencing an operational failure, or | |||
being attacked. In both cases, a client SHOULD retain the data for | being attacked. In both cases, a client SHOULD retain the data for | |||
later submission (subject to Private Browsing or other history- | later submission (subject to Private Browsing or other history- | |||
clearing actions taken by the user.) This is elaborated upon more in | clearing actions taken by the user.) This is elaborated upon more in | |||
Section 11.3. | Section 11.3. | |||
11.2. Proof Fetching Recommendations | 11.2. Proof Fetching Recommendations | |||
Proof fetching (both inclusion proofs and consistency proofs) should | Proof fetching (both inclusion proofs and consistency proofs) SHOULD | |||
be performed at random time intervals. If proof fetching occured all | be performed at random time intervals. If proof fetching occurred | |||
at once, in a flurry of activity, a log would know that SCTs or STHs | all at once, in a flurry of activity, a log would know that SCTs or | |||
recieved around the same time are more likely to come from a | STHs received around the same time are more likely to come from a | |||
particular client. While proof fetching is required to be done in a | particular client. While proof fetching is required to be done in a | |||
manner that attempts to be anonymous from the perspective of the log, | manner that attempts to be anonymous from the perspective of the log, | |||
the correlation of activity to a single client would still reveal | the correlation of activity to a single client would still reveal | |||
patterns of user behavior we wish to keep confidential. These | patterns of user behavior we wish to keep confidential. These | |||
patterns could be recognizable as a single user, or could reveal what | patterns could be recognizable as a single user, or could reveal what | |||
sites are commonly visited together in the aggregate. | sites are commonly visited together in the aggregate. | |||
[ TBD: What other recommendations do we want to make here? We can | ||||
talk more about the inadequecies of DNS... The first paragraph is 80% | ||||
identical between here and above ] | ||||
11.3. Record Distribution Recommendations | 11.3. Record Distribution Recommendations | |||
In several components of the CT Gossip ecosystem, the recommendation | In several components of the CT Gossip ecosystem, the recommendation | |||
is made that data from multiple sources be ingested, mixed, stored | is made that data from multiple sources be ingested, mixed, stored | |||
for an indeterminate period of time, provided (multiple times) to a | for an indeterminate period of time, provided (multiple times) to a | |||
third party, and eventually deleted. The instances of these | third party, and eventually deleted. The instances of these | |||
recommendations in this draft are: | recommendations in this draft are: | |||
o When a client receives SCTs during SCT Feedback, it should store | o When a client receives SCTs during SCT Feedback, it should store | |||
the SCTs and Certificate Chain for some amount of time, provide | the SCTs and Certificate Chain for some amount of time, provide | |||
skipping to change at page 32, line 18 ¶ | skipping to change at page 34, line 5 ¶ | |||
and each have options that may not be invoked. As one example, an | and each have options that may not be invoked. As one example, an | |||
HTTPS client should not mix SCTs from server A with SCTs from server | HTTPS client should not mix SCTs from server A with SCTs from server | |||
B and release server B's SCTs to Server A. As another example, an | B and release server B's SCTs to Server A. As another example, an | |||
HTTPS server may choose to resolve STHs to a single more current STH | HTTPS server may choose to resolve STHs to a single more current STH | |||
via proof fetching, but it is under no obligation to do so. | via proof fetching, but it is under no obligation to do so. | |||
These requirements should be met, but the general problem of | These requirements should be met, but the general problem of | |||
aggregating multiple pieces of data, choosing when and how many to | aggregating multiple pieces of data, choosing when and how many to | |||
release, and when to remove them is shared. This problem has | release, and when to remove them is shared. This problem has | |||
previously been considered in the case of Mix Networks and Remailers, | previously been considered in the case of Mix Networks and Remailers, | |||
including papers such as "From a Trickle to a Flood: Active Attacks | including papers such as [trickle]. | |||
on Several Mix Types", [Y], and [Z]. | ||||
There are several concerns to be addressed in this area, outlined | There are several concerns to be addressed in this area, outlined | |||
below. | below. | |||
11.3.1. Mixing Algorithm | 11.3.1. Mixing Algorithm | |||
When SCTs or STHs are recorded by a participant in CT Gossip and | When SCTs or STHs are recorded by a participant in CT Gossip and | |||
later used, it is important that they are selected from the datastore | later used, it is important that they are selected from the datastore | |||
in a non-deterministic fashion. | in a non-deterministic fashion. | |||
skipping to change at page 32, line 44 ¶ | skipping to change at page 34, line 30 ¶ | |||
the (encrypted) traffic to a server. When a client of interest | the (encrypted) traffic to a server. When a client of interest | |||
connects, the attacker makes a note. They observe more clients | connects, the attacker makes a note. They observe more clients | |||
connecting, and predicts at what point the client-of-interest's data | connecting, and predicts at what point the client-of-interest's data | |||
will be disclosed, and ensures that they query the server at that | will be disclosed, and ensures that they query the server at that | |||
point. | point. | |||
Although most important for servers, random ordering is still | Although most important for servers, random ordering is still | |||
strongly recommended for clients and Trusted Auditors. The above | strongly recommended for clients and Trusted Auditors. The above | |||
attack can still occur for these entities, although the circumstances | attack can still occur for these entities, although the circumstances | |||
are less straightforward. For clients, an attacker could observe | are less straightforward. For clients, an attacker could observe | |||
their behavior, note when they recieve an STH from a server, and use | their behavior, note when they receive an STH from a server, and use | |||
javascript to cause a network connection at the correct time to force | javascript to cause a network connection at the correct time to force | |||
a client to disclose the specific STH. Trusted Auditors are stewards | a client to disclose the specific STH. Trusted Auditors are stewards | |||
of sensitive client data. If an attacker had the ability to observe | of sensitive client data. If an attacker had the ability to observe | |||
the activities of a Trusted Auditor (perhaps by being a log, or | the activities of a Trusted Auditor (perhaps by being a log, or | |||
another auditor), they could perform the same attack - noting the | another auditor), they could perform the same attack - noting the | |||
disclosure of data from a client to the Trusted Auditor, and then | disclosure of data from a client to the Trusted Auditor, and then | |||
correlating a later disclosure from the Trusted Auditor as coming | correlating a later disclosure from the Trusted Auditor as coming | |||
from that client. | from that client. | |||
Random ordering can be ensured by several mechanisms. A datastore | Random ordering can be ensured by several mechanisms. A datastore | |||
can be shuffled, using a secure shuffling algorithm such as Fisher- | can be shuffled, using a secure shuffling algorithm such as Fisher- | |||
Yates. Alternately, a series of random indexes into the data store | Yates. Alternately, a series of random indexes into the data store | |||
can be selected (if a collision occurs, a new index is selected.) A | can be selected (if a collision occurs, a new index is selected.) A | |||
cryptographyically secure random number generator must be used in | cryptographically secure random number generator must be used in | |||
either case. If shuffling is performed, the datastore must be marked | either case. If shuffling is performed, the datastore must be marked | |||
'dirty' upon item insertion, and at least one shuffle operation | 'dirty' upon item insertion, and at least one shuffle operation | |||
occurs on a dirty datastore before data is retrieved from it for use. | occurs on a dirty datastore before data is retrieved from it for use. | |||
11.3.2. Flushing Attacks | 11.3.2. The Deletion Algorithm | |||
A flushing attack is an attempt by an adversary to flush a particular | ||||
piece of data from a pool. In the CT Gossip ecosystem, an attacker | ||||
may have performed an attack and left evidence of a compromised log | ||||
on a client or server. They would be interested in flushing that | ||||
data, i.e. tricking the target into gossiping or pollinating the | ||||
incriminating evidence with only attacker-controlled clients or | ||||
servers with the hope they trick the target into deleting it. | ||||
Servers are most vulnerable to flushing attacks, as they release | ||||
records to anonymous connections. An attacker can perform a Sybil | ||||
attack - connecting to the server hundreds or thousands of times in | ||||
an attempt to trigger repeated release of a record, and then | ||||
deletion. For this reason, servers must be especially aggressive | ||||
about retaining data for a longer period of time. | ||||
Clients are vulnerable to flushing attacks targetting STHs, as these | ||||
can be given to any cooperating server and an attacker can generally | ||||
induce connections to random servers using javascript. It would be | ||||
more difficult to perform a flushing attack against SCTs, as the | ||||
target server must be authenticated (and an attacker impersonating an | ||||
authentic server presents a recursive problem for the attacker). | ||||
Nonetheless, flushing SCTs should not be ruled impossible. A Trusted | ||||
Auditor may also be vulnerable to flushing attacks if it does not | ||||
perform auditing operations itself. | ||||
Flushing attacks are defended against using non-determinism and dummy | ||||
messages. The goal is to ensure that an adversary does not know for | ||||
certain if the data in question has been released or not, and if it | ||||
has been deleted or not. | ||||
[ TBD: At present, we do not have any support for dummy messages. Do | ||||
we want to define a dummy message that clients and servers alike know | ||||
to ignore? Will HTTP Compression leak the presence of >1 dummy | ||||
messages? | ||||
Is it sufficient to define a dummy message as _anything_ with an | ||||
invalid siganture? This would negatively impact SCT Feedback servers | ||||
that log all things just in case they're interesting. ] | ||||
11.3.3. The Deletion Algorithm | ||||
No entity in CT Gossip is required to delete SCTs or STHs at any | ||||
time, except to respect user's wishes such as private browsing mode | ||||
or clearing history. However, requiring infinite storage space is | ||||
not a desirable characteristic in a protocol, so deletion is | ||||
expected. | ||||
While deletion of SCTs and STHs will occur, proof fetching can ensure | ||||
that any misbehavior from a log will still be detected, even after | ||||
the direct evidence from the attack is deleted. Proof fetching | ||||
ensures that if a log presents a split view for a client, they must | ||||
maintain that split view in perpetuity. An inclusion proof from an | ||||
SCT to an STH does not erase the evidence - the new STH is evidence | ||||
itself. A consistency proof from that STH to a new one likewise - | ||||
the new STH is every bit as incriminating as the first. (Client | ||||
behavior in the situation where an SCT or STH cannot be resolved is | ||||
suggested in Section 11.1.2.) Because of this property, we recommend | ||||
that if a client is performing proof fetching, that they make every | ||||
effort to not delete an SCT or STH until it has been successfully | ||||
resolved to a new STH via a proof. | ||||
When it is time to delete a record, it is important that the decision | ||||
to do so not be done deterministicly. Introducing non-determinism in | ||||
the decision is absolutely necessary to prevent an adversary from | ||||
knowing with certainty that the record has been successfully flushed | ||||
from a target. Therefore, we speak of making a record 'eligible for | ||||
deletion' and then being processed by the 'deletion algorithm'. | ||||
Making a record eligible for deletion simply means that it will have | ||||
the deletion algorithm run. The deletion algorithm will use a | ||||
probability based system and a secure random number generator to | ||||
determine if the record will be deleted. | ||||
Although the deletion algorithm is specifically designed to be non- | No entity in CT Gossip is required to delete records at any time, | |||
deterministic, if the record has been resolved via proof to a new STH | except to respect user's wishes such as private browsing mode or | |||
the record may be safely deleted, as long as the new STH is retained. | clearing history. However, it is likely that over time the | |||
accumulated storage will grow in size and need to be pruned. | ||||
The actual deletion algorithm may be [STATISTICS HERE]. [ Something | While deletion of data will occur, proof fetching can ensure that any | |||
as simple as 'Pick an integer securely between 1 and 10. If it's | misbehavior from a log will still be detected, even after the direct | |||
greater than 7, delete the record.' Or something more complicated. ] | evidence from the attack is deleted. Proof fetching ensures that if | |||
a log presents a split view for a client, they must maintain that | ||||
split view in perpetuity. An inclusion proof from an SCT to an STH | ||||
does not erase the evidence - the new STH is evidence itself. A | ||||
consistency proof from that STH to a new one likewise - the new STH | ||||
is every bit as incriminating as the first. (Client behavior in the | ||||
situation where an SCT or STH cannot be resolved is suggested in | ||||
Section 11.1.2.) Because of this property, we recommend that if a | ||||
client is performing proof fetching, that they make every effort to | ||||
not delete data until it has been successfully resolved to a new STH | ||||
via a proof. | ||||
[ TODO Enumerating the problems of different types of mixes vs | When it is time to delete a record, it can be done in a way that | |||
Cottrell Mix ] | makes it more difficult for a successful flushing attack to to be | |||
performed. | ||||
11.3.3.1. Experimental Algorithms | 1. When the record cache has reached a certain size that is yet | |||
under the limit, aggressively perform proof fetching. This | ||||
should resolve records to a small set of STHs that can be | ||||
retained. Once a proof has been fetched, the record is safer to | ||||
delete. | ||||
More complex algorithms could be inserted at any step. Three | 2. If proof fetching has failed, or is disabled, begin by deleting | |||
examples are illustrated: | SCTs and Certificate Chains that have been successfully reported. | |||
Deletion from this set of SCTs should be done at random. For a | ||||
client, a submission is not counted as being reported unless it | ||||
is sent over a connection using a different SCT, so the attacker | ||||
is faced with a recursive problem. (For a server, this step does | ||||
not apply.) | ||||
SCTs are not eligible to be submitted to an Auditor of Last Resort. | 3. Attempt to save any submissions that have failed proof fetching | |||
Therefore, it is more important that they be resolved to STHs and | repeatedly, as these are the most likely to be indicative of an | |||
reported via SCT feedback. If fetching an inclusion proof regularly | attack. | |||
fails for a particular SCT, one can require it be reported more times | ||||
than normal via SCT Feedback before becoming eligible for deletion. | ||||
Before an item is made eligible for deletion by a client, the client | 4. Finally, if the above steps have been followed and have not | |||
could aim to make it difficult for a point-in-time attacker to flush | succeeded in reducing the size sufficiently, records may be | |||
the pool by not making an item eligible for deletion until the client | deleted at random. | |||
has moved networks (as seen by either the local IP address, or a | ||||
report-back providing the client with its observed public IP | ||||
address). The HTTPS client could also require reporting over a | ||||
timespan, e.g. it must be reported at least N time, M weeks apart. | ||||
This strategy could be employed always, or only when the client has | ||||
disabled proof fetching and the Auditor of Last Resort, as those two | ||||
mechanisms (when used together) will enable a client to report most | ||||
attacks. | ||||
11.3.3.2. Concrete Recommendations | Note that if proof fetching is disabled (which is expected although | |||
not required for servers) - the algorithm collapses down to 'delete | ||||
at random'. | ||||
The recommendations for behavior are: - If proof fetching is enabled, | The decision to delete records at random is intentional. Introducing | |||
do not delete an SCT until it has had a proof resolving it to an STH. | non-determinism in the decision is absolutely necessary to make it | |||
- If proof fetching continually fails for an SCT, do not make the | more difficult for an adversary to know with certainty or high | |||
item eligible for deletion of the SCT until it has been released, | confidence that the record has been successfully flushed from a | |||
multiple times, via SCT Feedback. - If proof fetching continually | target. | |||
fails for an STH, do not make the item eligible for deletion until it | ||||
has been queued for release to an Auditor of Last Resort. - Do not | ||||
dequeue entries to an Auditor of Last Resort if reporting fails. | ||||
Instead keep the items queued until they have been successfully sent. | ||||
- Use a probability based system, with a cryptographically secure | ||||
random number generator, to determine if an item should be deleted. | ||||
- Select items from the datastores by selecting random indexes into | ||||
the datastore. Use a cryptographically secure random number | ||||
generator. | ||||
[ TBD: More? ] | 11.4. Concrete Recommendations | |||
We present the following pseudocode as a concrete outline of our | We present the following pseudocode as a concrete outline of our | |||
suggestion. | policy recommendations. | |||
11.3.3.2.1. STH Data Structures | Both suggestions presented are applicable to both clients and | |||
servers. Servers may not perform proof fetching, in which case large | ||||
portions of the pseudocode are not applicable. But it should work in | ||||
either case. | ||||
11.4.1. STH Pollination | ||||
The STH class contains data pertaining specifically to the STH | The STH class contains data pertaining specifically to the STH | |||
itself. | itself. | |||
class STH | class STH | |||
{ | { | |||
uint32 proof_attempts | uint16 proof_attempts | |||
uint32 proof_failure_count | uint16 proof_failure_count | |||
uint32 num_reports_to_thirdparty | uint32 num_reports_to_thirdparty | |||
datetime timestamp | datetime timestamp | |||
byte[] data | byte[] data | |||
} | } | |||
The broader STH store itself would contain all the STHs known by an | The broader STH store itself would contain all the STHs known by an | |||
entity participating in STH Pollination (either client or server). | entity participating in STH Pollination (either client or server). | |||
This simplistic view of the class does not take into account the | This simplistic view of the class does not take into account the | |||
complicated locking that would likely be required for a data | complicated locking that would likely be required for a data | |||
structure being accessed by multiple threads. One thing to note | structure being accessed by multiple threads. Something to note | |||
about this pseudocode is that it aggressively removes STHs once they | about this pseudocode is that it does not remove STHs once they have | |||
have been resolved to a newer STH (if proof fetching is configured). | been resolved to a newer STH. Doing so might make older STHs within | |||
The only STHs in the store are ones that have never been resolved to | the validity window rarer and thus enable tracking. | |||
a newer STH, either because proof fetching does not occur, has | ||||
failed, or because the STH is considered too new to request a proof | ||||
for. It seems less likely that servers will perform proof fetching. | ||||
Therefore it would be recommended that the various constants in use | ||||
be increased considerably to ensure STHs are pollinated more | ||||
aggressively. | ||||
class STHStore | class STHStore | |||
{ | { | |||
STH[] sth_list | STH[] sth_list | |||
// This function is run after receiving a set of STHs from | // This function is run after receiving a set of STHs from | |||
// a third party in response to a pollination submission | // a third party in response to a pollination submission | |||
def insert(STH[] new_sths) { | def insert(STH[] new_sths) { | |||
foreach(new in new_sths) { | foreach(new in new_sths) { | |||
if(this.sth_list.contains(new)) | if(this.sth_list.contains(new)) | |||
continue | continue | |||
this.sth_list.insert(new) | this.sth_list.insert(new) | |||
} | } | |||
} | } | |||
// This function is called to possibly delete the given STH | // This function is called to delete the given STH | |||
// from the data store | ||||
def delete_maybe(STH s) { | ||||
//Perform statistical test and see if I should delete this bundle | ||||
} | ||||
// This function is called to (certainly) delete the given STH | ||||
// from the data store | // from the data store | |||
def delete_now(STH s) { | def delete_now(STH s) { | |||
this.sth_list.remove(s) | this.sth_list.remove(s) | |||
} | } | |||
// When it is time to perform STH Pollination, the HTTPS Client | // When it is time to perform STH Pollination, the HTTPS client | |||
// calls this function to get a selection of STHs to send as | // calls this function to get a selection of STHs to send as | |||
// feedback | // feedback | |||
def get_pollination_selection() { | def get_pollination_selection() { | |||
if(len(this.sth_list) < MAX_STH_TO_GOSSIP) | if(len(this.sth_list) < MAX_STH_TO_GOSSIP) | |||
return this.sth_list | return this.sth_list | |||
else { | else { | |||
indexes = set() | indexes = set() | |||
modulus = len(this.sth_list) | modulus = len(this.sth_list) | |||
while(len(indexes) < MAX_STH_TO_GOSSIP) { | while(len(indexes) < MAX_STH_TO_GOSSIP) { | |||
r = randomInt() % modulus | r = randomInt() % modulus | |||
// Ignore STHs that are past the validity window but not | ||||
// yet removed. | ||||
if(r not in indexes | if(r not in indexes | |||
&& now() - this.sth_list[i].timestamp < ONE_WEEK) | && now() - this.sth_list[i].timestamp < TWO_WEEKS) | |||
indexes.insert(r) | indexes.insert(r) | |||
} | } | |||
return_selection = [] | return_selection = [] | |||
foreach(i in indexes) { | foreach(i in indexes) { | |||
return_selection.insert(this.sth_list[i]) | return_selection.insert(this.sth_list[i]) | |||
} | } | |||
return return_selection | return return_selection | |||
} | } | |||
} | } | |||
} | } | |||
We also suggest a function that will be called periodically in the | ||||
We also suggest a function that can be called periodically in the | ||||
background, iterating through the STH store, performing a cleaning | background, iterating through the STH store, performing a cleaning | |||
operation and queuing consistency proofs. This function can live as | operation and queuing consistency proofs. This function can live as | |||
a member functions of the STHStore class. | a member functions of the STHStore class. | |||
def clean_list() { | //Just a suggestion: | |||
foreach(sth in this.sth_list) { | #define MIN_PROOF_FAILURES_CONSIDERED_SUSPICIOUS 3 | |||
if(now() - sth.timestamp > ONE_WEEK) { | def clean_list() { | |||
//STH is too old, we must remove it | foreach(sth in this.sth_list) { | |||
if(proof_fetching_enabled | ||||
&& auditor_of_last_resort_enabled | ||||
&& (sth.proof_failure_count / sth.proof_attempts) | ||||
> MIN_PROOF_FAILURE_RATIO_CONSIDERED_SUSPICIOUS) { | ||||
queue_sth_for_auditor_of_last_resort(sth) | ||||
delete_maybe(sth) | ||||
} else { | ||||
delete_now(sth) | ||||
} | ||||
} | ||||
else if(proof_fetching_enabled | if(now() - sth.timestamp > TWO_WEEKS) { | |||
&& now() - sth.timestamp > TWO_DAYS | //STH is too old, we must remove it | |||
&& now() - sth.timestamp > LOG_MMD) { | if(proof_fetching_enabled | |||
sth.proof_attempts++ | && auditor_of_last_resort_enabled | |||
queue_consistency_proof(sth, consistency_proof_callback) | && sth.proof_failure_count | |||
} | > MIN_PROOF_FAILURES_CONSIDERED_SUSPICIOUS) { | |||
} | queue_for_auditor_of_last_resort(sth, | |||
} | auditor_of_last_resort_callback) | |||
} else { | ||||
delete_now(sth) | ||||
} | ||||
} | ||||
11.3.3.2.2. STH Deletion Procedure | else if(proof_fetching_enabled | |||
&& now() - sth.timestamp > LOG_MMD | ||||
&& sth.proof_attempts != UINT16_MAX | ||||
// Only fetch a proof is we have never received a proof | ||||
// before. (This also avoids submitting something | ||||
// already in the queue.) | ||||
&& sth.proof_attempts == sth.proof_failure_count) { | ||||
sth.proof_attempts++ | ||||
queue_consistency_proof(sth, consistency_proof_callback) | ||||
} | ||||
} | ||||
} | ||||
The STH Deletion Procedure is run after successfully submitting a | These functions also exist in the STHStore class. | |||
list of STHs to a third party during pollination. The following | ||||
pseudocode would be included in the STHStore class, and called with | ||||
the result of get_pollination_selection(), after the STHs have been | ||||
(successfully) sent to the third party. | ||||
// This function is called after successfully pollinating STHs | // This function is called after successfully pollinating STHs | |||
// to a third party. It is passed the STHs sent to the third | // to a third party. It is passed the STHs sent to the third | |||
// party, which is the output of get_gossip_selection() | // party, which is the output of get_gossip_selection(), as well | |||
def after_submit_to_thirdparty(STH[] sth_list) | // as the STHs received in the response. | |||
{ | def successful_thirdparty_submission_callback(STH[] submitted_sth_list, | |||
foreach(sth in sth_list) | STH[] new_sths) | |||
{ | { | |||
sth.num_reports_to_thirdparty++ | foreach(sth in submitted_sth_list) { | |||
sth.num_reports_to_thirdparty++ | ||||
} | ||||
if(proof_fetching_enabled) { | this.insert(new_sths); | |||
if(now() - sth.timestamp > LOG_MMD) { | } | |||
sth.proof_attempts++ | ||||
queue_consistency_proof(sth, consistency_proof_callback) | ||||
} | ||||
if(auditor_of_last_resort_enabled | // Attempt auditor of last resort submissions until it succeeds | |||
&& sth.proof_failure_count > | def auditor_of_last_resort_callback(original_sth, error) { | |||
MIN_PROOF_ATTEMPTS_CONSIDERED_SUSPICIOUS | if(!error) { | |||
&& (sth.proof_failure_count / sth.proof_attempts) > | delete_now(original_sth) | |||
MIN_PROOF_FAILURE_RATIO_CONSIDERED_SUSPICIOUS) { | } | |||
queue_sth_for_auditor_of_last_resort(sth) | } | |||
} | ||||
} | ||||
else { //proof fetching not enabled | ||||
if(sth.num_reports_to_thirdparty | ||||
> MIN_STH_REPORTS_TO_THIRDPARTY) { | ||||
delete_maybe(sth) | ||||
} | ||||
} | ||||
} | ||||
} | ||||
def consistency_proof_callback(consistency_proof, | def consistency_proof_callback(consistency_proof, original_sth, error) { | |||
original_sth, | if(!error) { | |||
error) { | insert(consistency_proof.current_sth) | |||
if(!error) { | } else { | |||
insert(consistency_proof.current_sth) | original_sth.proof_failure_count++ | |||
delete_now(consistency_proof.original_sth) | } | |||
} else { | } | |||
original_sth.proof_failure_count++ | ||||
} | ||||
} | ||||
11.3.3.2.3. SCT Data Structures | 11.4.2. SCT Feedback | |||
TBD TBD This section is not well abstracted to be used for both | The SCT class contains data pertaining specifically to an SCT itself. | |||
servers and clients. TKTK | ||||
The SCT class contains data pertaining specifically to the SCT | ||||
itself. | ||||
class SCT | class SCT | |||
{ | { | |||
uint32 proof_attempts | uint16 proof_failure_count | |||
uint32 proof_failure_count | ||||
bool has_been_resolved_to_sth | bool has_been_resolved_to_sth | |||
bool proof_outstanding | ||||
byte[] data | byte[] data | |||
} | } | |||
The SCT bundle will contain the trusted certificate chain the HTTPS | The SCT bundle will contain the trusted certificate chain the HTTPS | |||
client built (chaining to a trusted root certificate.) It also | client built (chaining to a trusted root certificate.) It also | |||
contains the list of associated SCTs, the exact domain it is | contains the list of associated SCTs, the exact domain it is | |||
applicable to, and metadata pertaining to how often it has been | applicable to, and metadata pertaining to how often it has been | |||
reported to the third party. | reported to the third party. | |||
class SCTBundle | class SCTBundle | |||
skipping to change at page 41, line 38 ¶ | skipping to change at page 40, line 38 ¶ | |||
return true | return true | |||
} | } | |||
def insert_scts(sct[] sct_list) { | def insert_scts(sct[] sct_list) { | |||
this.sct_list.union(sct_list) | this.sct_list.union(sct_list) | |||
this.num_reports_to_thirdparty = 0 | this.num_reports_to_thirdparty = 0 | |||
} | } | |||
def has_been_fully_resolved_to_sths() { | def has_been_fully_resolved_to_sths() { | |||
foreach(s in this.sct_list) { | foreach(s in this.sct_list) { | |||
if(!s.has_been_resolved_to_sth) | if(!s.has_been_resolved_to_sth && !s.proof_outstanding) | |||
return false | return false | |||
} | } | |||
return true | return true | |||
} | } | |||
def max_proof_failure_count() { | def max_proof_failures() { | |||
uint32 max = 0 | uint max = 0 | |||
foreach(s in this.sct_list) { | foreach(sct in this.sct_list) { | |||
if(s.proof_failure_count > max) | if(sct.proof_failure_count > max) | |||
max = proof_failure_count | max = sct.proof_failure_count | |||
} | } | |||
return max | return max | |||
} | } | |||
} | } | |||
We suppose a large data structure is used, such as a hashmap, indexed | For each domain, we store a SCTDomainEntry that holds the SCTBundles | |||
by the domain name. For each domain, the structure will contain a | seen for that domain, as well as encapsulating some logic relating to | |||
data structure that holds the SCTBundles seen for that domain, as | SCT Feedback for that particular domain. In particular, this data | |||
well as encapsulating some logic relating to SCT Feedback for that | structure also contains the logic that handles domains not supporting | |||
particular domain. | SCT Feedback. Its behavior is: | |||
class SCTStore | 1. When a user visits a domain, SCT Feedback is attempted for it. | |||
{ | If it fails, it will retry after a month (configurable). If it | |||
string domain | succeeds, excellent. SCT Feedback data is still collected and | |||
datetime last_contact_for_domain | stored even if SCT Feedback failed. | |||
uint32 num_submissions_attempted | ||||
uint32 num_submissions_succeeded | ||||
SCTBundle[] observed_records | ||||
// This function is called after recieving an SCTBundle. | 2. After 3 month-long waits between failures, the domain will be | |||
// For Clients, this is after a successful connection to a | marked as failing long-term. No SCT Feedback data will be stored | |||
// HTTPS Server, calling this function with an SCTBundle | beyond meta-data, but SCT Feedback will still be attempted after | |||
// constructed from that certificate chain and SCTs | month-long waits | |||
// For Servers, this is after receiving SCT Feedback | ||||
def insert(SCTBundle b) { | ||||
if(operator_is_server) { | ||||
if(!passes_validity_checks(b)) | ||||
return | ||||
} | ||||
foreach(e in this.observed_records) { | ||||
if(e.equals(b)) | ||||
return | ||||
else if(e.approx_equals(b)) { | ||||
e.insert_scts(b.sct_list) | ||||
return | ||||
} | ||||
} | ||||
this.observed_records.insert(b) | ||||
} | ||||
// When it is time to perform SCT Feedback, the HTTPS Client | 3. If at any point in time, SCT Feedback succeeds, all failure | |||
// calls this function to get a selection of SCTBundles to send | counters are reset | |||
// as feedback | ||||
def get_gossip_selection() { | ||||
if(len(observed_records) > MAC_SCT_RECORDS_TO_GOSSIP) { | ||||
indexes = set() | ||||
modulus = len(observed_records) | ||||
while(len(indexes) < MAX_SCT_RECORDS_TO_GOSSIP) { | ||||
r = randomInt() % modulus | ||||
if(r not in indexes) | ||||
indexes.insert(r) | ||||
} | ||||
return_selection = [] | ||||
foreach(i in indexes) { | ||||
return_selection.insert(this.observed_records[i]) | ||||
} | ||||
return return_selection | 4. If a domain succeeds, but then begins failing, it must fail more | |||
} | than 90% of the time (configurable) and then the process begins | |||
else | at (2). | |||
return this.observed_records | ||||
} | ||||
def delete_maybe(SCTBundle b) { | If a domain is visited infrequently (say, once every 7 months) then | |||
//Perform statistical test and see if I should delete this bundle | it will be evicted from the cache and start all over again (according | |||
} | to the suggestion values in the below pseudocode). | |||
def delete_now(SCTBundle b) { | [ Note: To be certain the logic is correct I give the following test | |||
this.observed_records.remove(b) | cases which illustrate the intended behavior. Hopefully the code | |||
} | matches! | |||
def passes_validity_checks(SCTBundle b) { | Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
// This function performs the validity checks specified in | Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
// {{feedback-srvop}} | ... wait a month ... | |||
} | Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1 | |||
} | ... wait a month ... | |||
Succeed 1 month later num_submissions_attempted=13 num_submissions_succeeded=2 num_feedback_loop_failures=0(r) indicates (Reset) | ||||
-> Feedback is attempted regularly. | ||||
We also suggest a function that can be called periodically in the | Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
background, iterating through all SCTStore objects in the large | Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
hashmap (here called 'all_sct_stores') and removing old data. | ... wait a month ... | |||
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1 | ||||
... wait a month ... | ||||
Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2 | ||||
... wait a month ... | ||||
Succeed 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=0(r) | ||||
-> Feedback is attempted regularly. | ||||
def clear_old_data() | Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
{ | Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
foreach(storeEntry in all_sct_stores) | ... wait a month ... | |||
{ | Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1 | |||
if(storeEntry.num_submissions_succeeded == 0 | ... wait a month ... | |||
&& storeEntry.num_submissions_attempted | Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2 | |||
> MIN_SCT_ATTEMPTS_FOR_DOMAIN_TO_BE_IGNORED) | ... wait a month ... | |||
{ | Fail 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=3 | |||
all_sct_stores.remove(storeEntry) | ... clear_old_data() is run every hour ... | |||
} | num_submissions_attempted=0 num_submissions_succeeded=0 num_feedback_loop_failures=3 | |||
else if(storeEntry.num_submissions_succeeded > 0 | sct_feedback_failing_longterm=True | |||
&& now() - storeEntry.last_contact_for_domain | Fail 1 month later num_submissions_attempted=1 num_submissions_succeeded=0 num_feedback_loop_failures=4 | |||
> TIME_UNTIL_OLD_SCTDATA_ERASED) | sct_feedback_failing_longterm=True | |||
{ | ... clear_old_data() is run every hour ... | |||
all_sct_stores.remove(storeEntry) | num_submissions_attempted=0(r) num_submissions_succeeded=0 num_feedback_loop_failures=3 | |||
} | sct_feedback_failing_longterm=True | |||
} | Succeed 1 month later num_submissions_attempted=2 num_submissions_succeeded=1 num_feedback_loop_failures=0(r) | |||
} | sct_feedback_failing_longterm=False | |||
-> Feedback is attempted regularly. | ||||
11.3.3.2.4. SCT Deletion Procedure | Note above that the second run of clear_old_data() will reset num_submissions_attempted from 1 to 0. This is | |||
CRITICAL. Otherwise, we would have the below bug (where after 10 months of failures, a success would not hit | ||||
the required ratio to keep going) | ||||
The SCT Deletion procedure is more complicated than the respective | //The below represents a bug. | |||
STH procedure. This is because servers may elect not to participate | Succeed 1 Time num_submissions_attempted=1 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
in SCT Feedback, and this must be accounted for by being more | Fail 10 Times num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0 | |||
conservative in sending SCT reports to them. | ... wait a month ... | |||
Fail 1 month later num_submissions_attempted=12 num_submissions_succeeded=1 num_feedback_loop_failures=1 | ||||
... wait a month ... | ||||
Fail 1 month later num_submissions_attempted=13 num_submissions_succeeded=1 num_feedback_loop_failures=2 | ||||
... wait a month ... | ||||
Fail 1 month later num_submissions_attempted=14 num_submissions_succeeded=2 num_feedback_loop_failures=3 | ||||
... clear_old_data() is run every hour ... | ||||
num_submissions_attempted=0 num_submissions_succeeded=0 num_feedback_loop_failures=3 | ||||
sct_feedback_failing_longterm=True | ||||
Fail 1 month later num_submissions_attempted=1 num_submissions_succeeded=0 num_feedback_loop_failures=4 | ||||
sct_feedback_failing_longterm=True | ||||
Fail 9 times for 9 months | ||||
num_submissions_attempted=10 num_submissions_succeeded=0 num_feedback_loop_failures=13 | ||||
sct_feedback_failing_longterm=True | ||||
Succeed 1 month later num_submissions_attempted=11 num_submissions_succeeded=1 num_feedback_loop_failures=0(r) | ||||
sct_feedback_failing_longterm=False | ||||
-> Feedback is NOT attempted regularly. \] | ||||
The following pseudocode would be included in the SCTStore class, and | //Suggestions: | |||
called with the result of get_gossip_selection() after the SCT | // After concluding a domain doesn't support feedback, we try again | |||
Feedback has been sent (successfully) to the server. We also note | // after WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time to see if | |||
that the first experimental algorithm from above is included in the | // they added support | |||
pseudocode as an illustration. | #define WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS 1 month | |||
// This function is called after successfully providing SCT Feedback | // If we've waited MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE | |||
// to a server. It is passed the feedback sent to the server, which | // multiplied by WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, we | |||
// is the output of get_gossip_selection() | // still attempt SCT Feedback, but no longer bother storing any data | |||
def after_submit_to_thirdparty(SCTBundle[] submittedBundles) | // until the domain supports SCT Feedback | |||
{ | #define MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE 3 | |||
foreach(bundle in submittedBundles) | ||||
{ | ||||
bundle.num_reports_to_thirdparty++ | ||||
if(proof_fetching_enabled) { | // If this percentage of SCT Feedback attempts previously succeeded, | |||
if(!bundle.has_been_fully_resolved_to_sths()) { | // we consider the domain as supporting feedback and is just having | |||
// transient errors | ||||
#define MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING .10 | ||||
class SCTDomainEntry | ||||
{ | ||||
// This is the primary key of the object, the exact domain name it | ||||
// is valid for | ||||
string domain | ||||
// This is the last time the domain was contacted. For client | ||||
// operations it is updated whenever the client makes any request | ||||
// (not just feedback) to the domain. For server operations, it is | ||||
// updated whenever any client contacts the domain. Responsibility | ||||
// for updating lies OUTSIDE of the class | ||||
public datetime last_contact_for_domain | ||||
// This is the last time SCT Feedback was attempted for the domain. | ||||
// It is updated whenever feedback is attempted - responsibility for | ||||
// updating lies OUTSIDE of the class | ||||
// This is not used when this algorithm runs on servers | ||||
public datetime last_sct_feedback_attempt | ||||
// This is the number of times we have waited an | ||||
// WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS amount of time, and still failed | ||||
// e.g. 10 months of failures | ||||
// This is not used when this algorithm runs on servers | ||||
private uint16 num_feedback_loop_failures | ||||
// This is whether or not SCT Feedback has failed enough times that we | ||||
// should not bother storing data for it anymore. It is a small function | ||||
// used for illustrative purposes | ||||
// This is not used when this algorithm runs on servers | ||||
private bool sct_feedback_failing_longterm() | ||||
{ num_feedback_loop_failures >= MIN_SCT_FEEDBACK_ATTEMPTS_BEFORE_OMITTING_STORAGE } | ||||
// This is the number of SCT Feedback submissions attempted. | ||||
// Responsibility for incrementing lies OUTSIDE of the class | ||||
// (And watch for integer overflows) | ||||
// This is not used when this algorithm runs on servers | ||||
public uint16 num_submissions_attempted | ||||
// This is the number of successful SCT Feedback submissions. This | ||||
// variable is updated by the class. | ||||
// This is not used when this algorithm runs on servers | ||||
private uint16 num_submissions_succeeded | ||||
// This contains all the bundles of SCT data we have observed for | ||||
// this domain | ||||
SCTBundle[] observed_records | ||||
// This function can be called to determine if we should attempt | ||||
// SCT Feedback for this domain. | ||||
def should_attempt_feedback() { | ||||
// Servers always perform feedback! | ||||
if(operator_is_server) | ||||
return true | ||||
// If we have not tried in a month, try again | ||||
if(now() - last_sct_feedback_attempt > WAIT_BETWEEN_SCT_FEEDBACK_ATTEMPTS) | ||||
return true | ||||
// If we have tried recently, and it seems to be working, go for it! | ||||
if((num_submissions_succeeded / num_submissions_attempted) > | ||||
MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING) | ||||
return true | ||||
// Otherwise don't try | ||||
return false | ||||
} | ||||
// For Clients, this function is called after a successful | ||||
// connection to an HTTPS server, with a single SCTBundle | ||||
// constructed from that connection's certificate chain and SCTs. | ||||
// For Servers, this is called after receiving SCT Feedback with | ||||
// all the bundles sent in the feedback. | ||||
def insert(SCTBundle[] bundles) { | ||||
// Do not store data for long-failing domains | ||||
if(sct_feedback_failing_longterm()) { | ||||
return | ||||
} | ||||
foreach(b in bundles) { | ||||
if(operator_is_server) { | ||||
if(!passes_validity_checks(b)) | ||||
return | ||||
} | ||||
bool have_inserted = false | ||||
foreach(e in this.observed_records) { | ||||
if(e.equals(b)) | ||||
return | ||||
else if(e.approx_equals(b)) { | ||||
have_inserted = true | ||||
e.insert_scts(b.sct_list) | ||||
} | ||||
} | ||||
if(!have_inserted) | ||||
this.observed_records.insert(b) | ||||
} | ||||
SCTStoreManager.update_cache_percentage() | ||||
} | ||||
// When it is time to perform SCT Feedback, the HTTPS client | ||||
// calls this function to get a selection of SCTBundles to send | ||||
// as feedback | ||||
def get_gossip_selection() { | ||||
if(len(observed_records) > MAX_SCT_RECORDS_TO_GOSSIP) { | ||||
indexes = set() | ||||
modulus = len(observed_records) | ||||
while(len(indexes) < MAX_SCT_RECORDS_TO_GOSSIP) { | ||||
r = randomInt() % modulus | ||||
if(r not in indexes) | ||||
indexes.insert(r) | ||||
} | ||||
return_selection = [] | ||||
foreach(i in indexes) { | ||||
return_selection.insert(this.observed_records[i]) | ||||
} | ||||
return return_selection | ||||
} | ||||
else | ||||
return this.observed_records | ||||
} | ||||
def passes_validity_checks(SCTBundle b) { | ||||
// This function performs the validity checks specified in | ||||
// {{feedback-srvop}} | ||||
} | ||||
} | ||||
The SCTDomainEntry is responsible for handling the outcome of a | ||||
submission report for that domain using its member function: | ||||
// This function is called after providing SCT Feedback | ||||
// to a server. It is passed the feedback sent to the other party, which | ||||
// is the output of get_gossip_selection(), and also the SCTBundle | ||||
// representing the connection the data was sent on. | ||||
// (When this code runs on the server, connectionBundle is NULL) | ||||
// If the Feedback was not sent successfully, error is True | ||||
def after_submit_to_thirdparty(error, SCTBundle[] submittedBundles, | ||||
SCTBundle connectionBundle) | ||||
{ | ||||
// Server operation in this instance is exceedingly simple | ||||
if(operator_is_server) { | ||||
if(error) | ||||
return | ||||
foreach(bundle in submittedBundles) | ||||
bundle.num_reports_to_thirdparty++ | ||||
return | ||||
} | ||||
// Client behavior is much more complicated | ||||
if(error) { | ||||
if(sct_feedback_failing_longterm()) { | ||||
num_feedback_loop_failures++ | ||||
} | ||||
else if((num_submissions_succeeded / num_submissions_attempted) | ||||
> MIN_RATIO_FOR_SCT_FEEDBACK_TO_BE_WORKING) { | ||||
// Do nothing. num_submissions_succeeded will not be incremented | ||||
// After enough of these failures, the ratio will fall beyond | ||||
// acceptable | ||||
} else { | ||||
// The domain has begun its three-month grace period. We will | ||||
// attempt submissions once a month | ||||
num_feedback_loop_failures++ | ||||
} | ||||
return | ||||
} | ||||
// We succeeded, so reset all of our failure states | ||||
// Note, there is a race condition here if clear_old_data() is called | ||||
// while this callback is outstanding. | ||||
num_feedback_loop_failures = 0 | ||||
if(num_submissions_succeeded != UINT16_MAX ) | ||||
num_submissions_succeeded++ | ||||
foreach(bundle in submittedBundles) | ||||
{ | ||||
// Compare Certificate Chains, if they do not match, it counts as a | ||||
// submission. | ||||
if(!connectionBundle.approx_equals(bundle)) | ||||
bundle.num_reports_to_thirdparty++ | ||||
else { | ||||
// This check ensures that a SCT Bundle is not considered reported | ||||
// if it is submitted over a connection with the same SCTs. This | ||||
// satisfies the constraint in Paragraph 5 of {{feedback-clisrv}} | ||||
// Consider three submission scenarios: | ||||
// Submitted SCTs Connection SCTs Considered Submitted | ||||
// A, B A, B No - no new information | ||||
// A A, B Yes - B is a new SCT | ||||
// A, B A No - no new information | ||||
if(connectionBundle.sct_list is NOT a subset of bundle.sct_list) | ||||
bundle.num_reports_to_thirdparty++ | ||||
} | ||||
} | ||||
} | ||||
Instances of the SCTDomainEntry class are stored as part of a larger | ||||
class that manages the entire SCT Cache, storing them in a hashmap | ||||
keyed by domain. This class also tracks the current size of the | ||||
cache, and will trigger cache eviction. | ||||
//Suggestions: | ||||
#define CACHE_PRESSURE_SAFE .50 | ||||
#define CACHE_PRESSURE_IMMINENT .70 | ||||
#define CACHE_PRESSURE_ALMOST_FULL .85 | ||||
#define CACHE_PRESSURE_FULL .95 | ||||
#define WAIT_BETWEEN_IMMINENT_CACHE_EVICTION 5 minutes | ||||
class SCTStoreManager | ||||
{ | ||||
hashmap<String, SCTDomainEntry> all_sct_entries | ||||
uint32 current_cache_size | ||||
datetime imminent_cache_pressure_check_performed | ||||
float current_cache_percentage() { | ||||
return current_cache_size / MAX_CACHE_SIZE; | ||||
} | ||||
static def update_cache_percentage() { | ||||
// This function calculates the current size of the cache | ||||
// and updates current_cache_size | ||||
/* ... perform calculations ... */ | ||||
current_cache_size = /* new calculated value */ | ||||
// Perform locking to prevent multiple of these functions being | ||||
// called concurrently or unnecessarily | ||||
if(current_cache_percentage() > CACHE_PRESSURE_FULL) { | ||||
cache_is_full() | ||||
} | ||||
else if(current_cache_percentage() > CACHE_PRESSURE_ALMOST_FULL) { | ||||
cache_pressure_almost_full() | ||||
} | ||||
else if(current_cache_percentage() > CACHE_PRESSURE_IMMINENT) { | ||||
// Do not repeatedly perform the imminent cache pressure operation | ||||
if(now() - imminent_cache_pressure_check_performed > | ||||
WAIT_BETWEEN_IMMINENT_CACHE_EVICTION) { | ||||
cache_pressure_is_imminent() | ||||
} | ||||
} | ||||
} | ||||
} | ||||
The SCTStoreManager contains a function that will be called | ||||
periodically in the background, iterating through all SCTDomainEntry | ||||
objects and performing maintenance tasks. It removes data for | ||||
domains we have not contacted in a long time. This function is not | ||||
intended to clear data if the cache is getting full, separate | ||||
functions are used for that. | ||||
// Suggestions: | ||||
#define TIME_UNTIL_OLD_SUBMITTED_SCTDATA_ERASED 3 months | ||||
#define TIME_UNTIL_OLD_UNSUBMITTED_SCTDATA_ERASED 6 months | ||||
def clear_old_data() | ||||
{ | ||||
foreach(domainEntry in all_sct_stores) | ||||
{ | ||||
// Queue proof fetches | ||||
if(proof_fetching_enabled) { | ||||
foreach(sctBundle in domainEntry.observed_records) { | ||||
if(!sctBundle.has_been_fully_resolved_to_sths()) { | ||||
foreach(s in bundle.sct_list) { | foreach(s in bundle.sct_list) { | |||
if(!s.has_been_resolved_to_sth) { | if(!s.has_been_resolved_to_sth && !s.proof_outstanding) { | |||
s.proof_attempts++ | sct.proof_outstanding = True | |||
queue_inclusion_proof(sct, inclusion_proof_callback) | queue_inclusion_proof(sct, inclusion_proof_callback) | |||
} | } | |||
} | } | |||
} | } | |||
else { | ||||
if(run_ct_gossip_experiment_one) { | ||||
if(bundle.num_reports_to_thirdparty | ||||
> MIN_SCT_REPORTS_TO_THIRDPARTY | ||||
&& bundle.num_reports_to_thirdparty * 1.5 | ||||
> bundle.max_proof_failure_count()) { | ||||
maybe_delete(bundle) | ||||
} | ||||
} | ||||
else { //Do not run experiment | ||||
if(bundle.num_reports_to_thirdparty | ||||
> MIN_SCT_REPORTS_TO_THIRDPARTY) { | ||||
maybe_delete(bundle) | ||||
} | ||||
} | ||||
} | ||||
} | ||||
else {//proof fetching not enabled | ||||
if(bundle.num_reports_to_thirdparty | ||||
> (MIN_SCT_REPORTS_TO_THIRDPARTY | ||||
* NO_PROOF_FETCHING_REPORT_INCREASE_FACTOR)) { | ||||
maybe_delete(bundle) | ||||
} | ||||
} | } | |||
} | } | |||
} | ||||
// This function is a callback invoked after an inclusion proof | // Do not store data for domains who are not supporting SCT | |||
// has been retrieved | if(!operator_is_server | |||
def inclusion_proof_callback(inclusion_proof, original_sct, error) | && domainEntry.sct_feedback_failing_longterm()) | |||
{ | { | |||
if(!error) { | // Note that reseting these variables every single time is | |||
original_sct.has_been_resolved_to_sth = True | // necessary to avoid a bug | |||
insert_to_sth_datastore(inclusion_proof.new_sth) | all_sct_stores[domainEntry].num_submissions_attempted = 0 | |||
} else { | all_sct_stores[domainEntry].num_submissions_succeeded = 0 | |||
original_sct.proof_failure_count++ | delete all_sct_stores[domainEntry].observed_records | |||
all_sct_stores[domainEntry].observed_records = NULL | ||||
} | } | |||
// This check removes successfully submitted data for | ||||
// old domains we have not dealt with in a long time | ||||
if(domainEntry.num_submissions_succeeded > 0 | ||||
&& now() - domainEntry.last_contact_for_domain | ||||
> TIME_UNTIL_OLD_SUBMITTED_SCTDATA_ERASED) | ||||
{ | ||||
all_sct_stores.remove(domainEntry) | ||||
} | ||||
// This check removes unsuccessfully submitted data for | ||||
// old domains we have not dealt with in a very long time | ||||
if(now() - domainEntry.last_contact_for_domain | ||||
> TIME_UNTIL_OLD_UNSUBMITTED_SCTDATA_ERASED) | ||||
{ | ||||
all_sct_stores.remove(domainEntry) | ||||
} | ||||
SCTStoreManager.update_cache_percentage() | ||||
} | ||||
Inclusion Proof Fetching is handled fairly independently | ||||
// This function is a callback invoked after an inclusion proof | ||||
// has been retrieved. It can exist on the SCT class or independently, | ||||
// so long as it can modify the SCT class' members | ||||
def inclusion_proof_callback(inclusion_proof, original_sct, error) | ||||
{ | ||||
// Unlike the STH code, this counter must be incremented on the | ||||
// callback as there is a race condition on using this counter in the | ||||
// cache_* functions. | ||||
original_sct.proof_attempts++ | ||||
original_sct.proof_outstanding = False | ||||
if(!error) { | ||||
original_sct.has_been_resolved_to_sth = True | ||||
insert_to_sth_datastore(inclusion_proof.new_sth) | ||||
} else { | ||||
original_sct.proof_failure_count++ | ||||
} | } | |||
} | ||||
If the cache is getting full, these three member functions of the | ||||
SCTStoreManager class will be used. | ||||
// ----------------------------------------------------------------- | ||||
// This function is called when the cache is not yet full, but is | ||||
// nearing it. It prioritizes deleting data that should be safe | ||||
// to delete (because it has been shared with the site or resolved | ||||
// to a STH) | ||||
def cache_pressure_is_imminent() | ||||
{ | ||||
bundlesToDelete = [] | ||||
foreach(domainEntry in all_sct_stores) { | ||||
foreach(sctBundle in domainEntry.observed_records) { | ||||
if(proof_fetching_enabled) { | ||||
// First, queue proofs for anything not already queued. | ||||
if(!sctBundle.has_been_fully_resolved_to_sths()) { | ||||
foreach(sct in bundle.sct_list) { | ||||
if(!sct.has_been_resolved_to_sth | ||||
&& !sct.proof_outstanding) { | ||||
sct.proof_outstanding = True | ||||
queue_inclusion_proof(sct, inclusion_proof_callback) | ||||
} | ||||
} | ||||
} | ||||
// Second, consider deleting entries that have been fully | ||||
// resolved. | ||||
else { | ||||
bundlesToDelete.append( Struct(domainEntry, sctBundle) ) | ||||
} | ||||
} | ||||
// Third, consider deleting entries that have been successfully | ||||
// reported | ||||
if(sctBundle.num_reports_to_thirdparty > 0) { | ||||
bundlesToDelete.append( Struct(domainEntry, sctBundle) ) | ||||
} | ||||
} | ||||
} | ||||
// Third, delete the eligible entries at random until the cache is | ||||
// at a safe level | ||||
uint recalculateIndex = 0 | ||||
#define RECALCULATE_EVERY_N_OPERATIONS 50 | ||||
while(bundlesToDelete.length > 0 && | ||||
current_cache_percentage() > CACHE_PRESSURE_SAFE) { | ||||
uint rndIndex = rand() % bundlesToDelete.length | ||||
bundlesToDelete[rndIndex].domainEntry.observed_records.remove(bundlesToDelete[rndIndex].sctBundle) | ||||
bundlesToDelete.removeAt(rndIndex) | ||||
recalculateIndex++ | ||||
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { | ||||
update_cache_percentage() | ||||
} | ||||
} | ||||
// Finally, tell the proof fetching engine to go faster | ||||
if(proof_fetching_enabled) { | ||||
// This function would speed up proof fetching until an | ||||
// arbitrary time has passed. Perhaps until it has fetched | ||||
// proofs for the number of items currently in its queue? Or | ||||
// a percentage of them? | ||||
proof_fetch_faster_please() | ||||
} | ||||
update_cache_percentage(); | ||||
} | ||||
// ----------------------------------------------------------------- | ||||
// This function is called when the cache is almost full. It will | ||||
// evict entries at random, while attempting to save entries that | ||||
// appear to have proof fetching failures | ||||
def cache_pressure_almost_full() | ||||
{ | ||||
uint recalculateIndex = 0 | ||||
uint savedRecords = 0 | ||||
#define RECALCULATE_EVERY_N_OPERATIONS 50 | ||||
while(all_sct_stores.length > savedRecords && | ||||
current_cache_percentage() > CACHE_PRESSURE_SAFE) { | ||||
uint rndIndex1 = rand() % all_sct_stores.length | ||||
uint rndIndex2 = rand() % all_sct_stores[rndIndex1].observed_records.length | ||||
if(proof_fetching_enabled) { | ||||
if(all_sct_stores[rndIndex1].observed_records[rndIndex2].max_proof_failures() > | ||||
MIN_PROOF_FAILURES_CONSIDERED_SUSPICIOUS) { | ||||
savedRecords++ | ||||
continue | ||||
} | ||||
} | ||||
// If proof fetching is not enabled we need some other logic | ||||
else { | ||||
if(sctBundle.num_reports_to_thirdparty == 0) { | ||||
savedRecords++ | ||||
continue | ||||
} | ||||
} | ||||
all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2) | ||||
if(all_sct_stores[rndIndex1].observed_records.length == 0) { | ||||
all_sct_stores.removeAt(rndIndex1) | ||||
} | ||||
recalculateIndex++ | ||||
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { | ||||
update_cache_percentage() | ||||
} | ||||
} | ||||
update_cache_percentage(); | ||||
} | ||||
// ----------------------------------------------------------------- | ||||
// This function is called when the cache is full, and will evict | ||||
// cache entries at random | ||||
def cache_is_full() | ||||
{ | ||||
uint recalculateIndex = 0 | ||||
#define RECALCULATE_EVERY_N_OPERATIONS 50 | ||||
while(all_sct_stores.length > 0 && | ||||
current_cache_percentage() > CACHE_PRESSURE_SAFE) { | ||||
uint rndIndex1 = rand() % all_sct_stores.length | ||||
uint rndIndex2 = rand() % all_sct_stores[rndIndex1].observed_records.length | ||||
all_sct_stores[rndIndex1].observed_records.removeAt(rndIndex2) | ||||
if(all_sct_stores[rndIndex1].observed_records.length == 0) { | ||||
all_sct_stores.removeAt(rndIndex1) | ||||
} | ||||
recalculateIndex++ | ||||
if(recalculateIndex % RECALCULATE_EVERY_N_OPERATIONS == 0) { | ||||
update_cache_percentage() | ||||
} | ||||
} | ||||
update_cache_percentage(); | ||||
} | ||||
12. IANA considerations | 12. IANA considerations | |||
[ TBD ] | [ TBD ] | |||
13. Contributors | 13. Contributors | |||
The authors would like to thank the following contributors for | The authors would like to thank the following contributors for | |||
valuable suggestions: Al Cutter, Ben Laurie, Benjamin Kaduk, Josef | valuable suggestions: Al Cutter, Ben Laurie, Benjamin Kaduk, Josef | |||
Gustafsson, Karen Seo, Magnus Ahltorp, Steven Kent, Yan Zhu. | Gustafsson, Karen Seo, Magnus Ahltorp, Steven Kent, Yan Zhu. | |||
14. ChangeLog | 14. ChangeLog | |||
14.1. Changes between ietf-01 and ietf-02 | 14.1. Changes between ietf-02 and ietf-03 | |||
o TBD's resolved. | ||||
o References added. | ||||
o Pseduocode changed to work for both clients and servers. | ||||
14.2. Changes between ietf-01 and ietf-02 | ||||
o Requiring full certificate chain in SCT Feedback. | o Requiring full certificate chain in SCT Feedback. | |||
o Clarifications on what clients store for and send in SCT Feedback | o Clarifications on what clients store for and send in SCT Feedback | |||
added. | added. | |||
o SCT Feedback server operation updated to protect against DoS | o SCT Feedback server operation updated to protect against DoS | |||
attacks on servers. | attacks on servers. | |||
o Pre-Loaded vs Locally Added Anchors explained. | o Pre-Loaded vs Locally Added Anchors explained. | |||
o Base for well-known URL's changed. | o Base for well-known URL's changed. | |||
o Remove all mentions of monitors - gossip deals with adutitors. | o Remove all mentions of monitors - gossip deals with auditors. | |||
o New sections added: Trusted Auditor protocol, attacks by actively | o New sections added: Trusted Auditor protocol, attacks by actively | |||
malicious log, the Dual-CA compromise attack, policy | malicious log, the Dual-CA compromise attack, policy | |||
recommendations, | recommendations, | |||
14.2. Changes between ietf-00 and ietf-01 | 14.3. Changes between ietf-00 and ietf-01 | |||
o Improve langugage and readability based on feedback from Stephen | o Improve language and readability based on feedback from Stephen | |||
Kent. | Kent. | |||
o STH Pollination Proof Fetching defined and indicated as optional. | o STH Pollination Proof Fetching defined and indicated as optional. | |||
o 3-Method Ecosystem section added. | o 3-Method Ecosystem section added. | |||
o Cases with Logs ceasing operation handled. | o Cases with Logs ceasing operation handled. | |||
o Text on tracking via STH Interaction added. | o Text on tracking via STH Interaction added. | |||
o Section with some early recommendations for mixing added. | o Section with some early recommendations for mixing added. | |||
o Section detailing blocking connections, frustrating it, and the | o Section detailing blocking connections, frustrating it, and the | |||
implications added. | implications added. | |||
14.3. Changes between -01 and -02 | 14.4. Changes between -01 and -02 | |||
o STH Pollination defined. | o STH Pollination defined. | |||
o Trusted Auditor Relationship defined. | o Trusted Auditor Relationship defined. | |||
o Overview section rewritten. | o Overview section rewritten. | |||
o Data flow picture added. | o Data flow picture added. | |||
o Section on privacy considerations expanded. | o Section on privacy considerations expanded. | |||
14.4. Changes between -00 and -01 | 14.5. Changes between -00 and -01 | |||
o Add the SCT feedback mechanism: Clients send SCTs to originating | o Add the SCT feedback mechanism: Clients send SCTs to originating | |||
web server which shares them with auditors. | web server which shares them with auditors. | |||
o Stop assuming that clients see STHs. | o Stop assuming that clients see STHs. | |||
o Don't use HTTP headers but instead .well-known URL's - avoid that | o Don't use HTTP headers but instead .well-known URL's - avoid that | |||
battle. | battle. | |||
o Stop referring to trans-gossip and trans-gossip-transport-https - | o Stop referring to trans-gossip and trans-gossip-transport-https - | |||
skipping to change at page 47, line 27 ¶ | skipping to change at page 55, line 42 ¶ | |||
Laurie, B., Langley, A., Kasper, E., Messeri, E., and R. | Laurie, B., Langley, A., Kasper, E., Messeri, E., and R. | |||
Stradling, "Certificate Transparency", October 2015, | Stradling, "Certificate Transparency", October 2015, | |||
<https://datatracker.ietf.org/doc/draft-ietf-trans- | <https://datatracker.ietf.org/doc/draft-ietf-trans- | |||
rfc6962-bis/>. | rfc6962-bis/>. | |||
[RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data | [RFC7159] Bray, T., "The JavaScript Object Notation (JSON) Data | |||
Interchange Format", RFC 7159, March 2014. | Interchange Format", RFC 7159, March 2014. | |||
15.2. Informative References | 15.2. Informative References | |||
[double-keying] | ||||
Perry, M., Clark, E., and S. Murdoch, "Cross-Origin | ||||
Identifier Unlinkability", May 2015, | ||||
<https://www.torproject.org/projects/torbrowser/ | ||||
design/#identifier-linkability>. | ||||
[draft-ct-over-dns] | ||||
Laurie, B., Phaneuf, P., and A. Eijdenberg, "Certificate | ||||
Transparency over DNS", February 2016, | ||||
<https://github.com/google/certificate-transparency- | ||||
rfcs/blob/master/dns/draft-ct-over-dns.md>. | ||||
[draft-ietf-trans-threat-analysis-03] | [draft-ietf-trans-threat-analysis-03] | |||
Kent, S., "Attack Model and Threat for Certificate | Kent, S., "Attack Model and Threat for Certificate | |||
Transparency", October 2015, | Transparency", October 2015, | |||
<https://datatracker.ietf.org/doc/draft-ietf-trans-threat- | <https://datatracker.ietf.org/doc/draft-ietf-trans-threat- | |||
analysis/>. | analysis/>. | |||
[dual-ca-compromise-attack] | ||||
Gillmor, D., "can CT defend against dual CA compromise?", | ||||
n.d., <https://www.ietf.org/mail- | ||||
archive/web/trans/current/msg01984.html>. | ||||
[gossip-mixing] | ||||
Ritter, T., "A Bit on Certificate Transparency Gossip", | ||||
June 2016, <https://ritter.vg/blog- | ||||
a_bit_on_certificate_transparency_gossip.html>. | ||||
[trickle] Serjantov, A., Dingledine, R., and . Paul Syverson, "From | ||||
a Trickle to a Flood: Active Attacks on Several Mix | ||||
Types", October 2002, | ||||
<http://freehaven.net/doc/batching-taxonomy/taxonomy.pdf>. | ||||
Authors' Addresses | Authors' Addresses | |||
Linus Nordberg | Linus Nordberg | |||
NORDUnet | NORDUnet | |||
Email: linus@nordu.net | Email: linus@nordu.net | |||
Daniel Kahn Gillmor | Daniel Kahn Gillmor | |||
ACLU | ACLU | |||
End of changes. 187 change blocks. | ||||
599 lines changed or deleted | 1049 lines changed or added | |||
This html diff was produced by rfcdiff 1.45. The latest version is available from http://tools.ietf.org/tools/rfcdiff/ |