2017 Online Trust Audit Methodology - January 2017
Briefing Deck (Updated March 6, 2017)
The 2017 Online Trust Audit is the 9th year OTA will be conducting an independent analysis and benchmark report of the adoption of security standards and responsible privacy practices. This methodology reflects comments received in response to OTA's public call for comments in September 2016, commonly accepted best practices and emerging threat vectors.
The 2017 Audit will evaluate approximately 1,000 websites across multiple sectors including; the Internet Retailer Top 500, the Top 100 Banks, the Top 100 Consumer sites, the Top 50 Federal Government sites, the Top 100 News/ Media sites and OTA Members. New for 2017 is the addition of a new sector including the top 100 ISPs, Hosters and Mailbox providers.
Sites are eligible to receive 300 total base points, including up to 100 points in each category. Bonus points are available for implementation of emerging best practices and penalties are assessed for vulnerabilities, breaches and regulatory settlements. The 2017 scoring has been expanded and enhanced with additional weight and granularity given to key practices.
To qualify for the Honor Roll status, sites need to receive a composite score of 80% or better and a score of at least 60 in each of the three separate categories. Each sector will be scored in three categories:
- Domain, Brand & Consumer Protection
- Site, Server & Infrastructure Security
- Privacy, Transparency & Disclosures
The Audit is planned to be completed between mid-April and the end of May, 2017. In total, it is estimated that more than 500 million email headers and approximately 100,000 web pages will be analyzed. With the goal to drive adoption and awareness of best practices, allowing all companies the ability to access their status and optimize their scores, OTA announced the criteria in late January 2017 via press release and on the OTA website, blog and external facing newsletters.
It should be noted that this research is based on a “slice of time” and individual companies may have adopted or change their security and privacy practices after the Audit. OTA recognizes that the sites examined might be using other technologies (which our tools or research did not detect) to authenticate domains or subdomains, secure their infrastructures, track users on their sites, etc. Due to the sensitivity of this data and risk of disclosing vulnerabilities, individual organization’s scores and data will not be publicly available. Information will be provided to site owners upon written request and verification. For details, including reporting fees, please send an email to admin @ otalliance.org.
Email Authentication (SPF, DKIM & DMARC) – The report will analyze more than 500 million emails and the respective DNS infrastructure of leading sites and subdomains. Email authentication assesses efforts to protect users from domain and email spoofing via the adoption of two industry leading protocols – Sender Policy Framework (SPF) and Domain-Keys Identified Mail (DKIM). Sites receive a maximum of 100 points by 1) implementing both SPF and DKIM authentication at the top level domain (e.g., yourdomain.com) as well as on their respective subdomains (e.g., email.yourdomain.com), and 2) implementing a DMARC record with a “reject” policy. Verification of DKIM-signed email requires review of the email headers of individual emails via data sampling providing by Agari, Microsoft and other data providers. Augmenting previous years’ methodology, OTA subscribes to email newsletters and/or submits inquiries to sites to review responses, which provides increased granularity of email data.
Results are integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll. New in 2017, invalid SPF and DMARC records will not receive credit. Likewise, "naked" DMARC records (a policy of "none" with no reporting) will not receive credit. Verification of SPF and DMARC records will be completed using the OTA DNS record lookup tool and additional analysis conducted by ValiMail and other OTA members.
Domain Locking – Domain locking is a security enhancement offered by most registrars to help prevent unauthorized transfers of your domain to another registrar or web host by locking your domain name servers. When your domain is locked, you'll be substantially protected from unauthorized third parties who might try to redirect your name servers or transfer your domain without your permission. Sites receive a penalty if their domain is not locked.
Transport Layered Security (TLS) for Email – Sites which implement "opportunistic TLS" will receive bonus points. TLS helps prevent eavesdropping on email as it moves between email servers that have enabled TLS protections for email. Just as TLS can be used to secure web communications (HTTPS), it can secure email transport. To maximize the content security and privacy, TLS is required between all the servers that handle the message including hops between internal and external servers. TLS adoption will be assessed using TLS databases provided by Twitter, Google and others as well as examination of email received from audited entities.
IPv6 & Domain Name System Security Extension (DNSSEC) – Testing will be completed using public tools and browser plug-ins, including data provided by Verisign and Infoblox. Sites adopting IPv6 and/or DNSSEC will receive bonus points.
(New in 2017) Multi-Factor Authentication – Sites which provide an option for multi-factor authentication will receive bonus points. By judiciously combining a strong password with additional factors, such as a fingerprint or a single use code delivered in a text message, accounts are better protected from account takeovers and unauthorized password resets.
Server and SSL Configuration – Sites will be evaluated using a combination of data and tools from DigiCert, High-Tech Bridge SA, Qualys Labs, Security Scorecard, SiteLock and Symantec. These tools provide visibility into the server architecture, configuration and digital certificates. Testing checks for weak keys, protocols, algorithms and server misconfigurations that can enable attackers to exploit system vulnerabilities and compromise SSL communications. Sites are also examined for application and network security, IP reputation and patching cadence. A blended score from the results of these tools makes up the 100 baseline points in this category.
Organization Validation (OV) and Domain Validated (DV) certificates receive neutral scoring. Organization Validation (OV) and Extended Validation (EV) certificates contain the verified name of the entity that controls the website. Certificate authorities (CAs) issuing these certificates check with third parties to establish the official name of the organization and where they are located. By contrast, Domain-Validated certificates are typically verified through automated processes. A DV certificate contains no identifying information in the organization name field. Typically, this value just re-states the domain name or simply says "Not Validated." Although the certificate supports transaction encryption, the end-user cannot confirm the identity of the organization on the other end.
Extended Validation SSL Certificates (EV SSL) – EV SSL offers visible confirmation of site identity to the user. The 2017 analysis will focus on all sites with SSL connections, not limiting the evaluation to consumer facing e-commerce or banking sites. Cybercriminals target business-to-business, social networking and government sector sites with non-EV Certificates. Acquiring an Extended Validation certificate requires extensive verification by the certificate authority. Sites with EV SSL Certificates receive bonus points.
Always On SSL (AOSSL) – Sites are evaluated for the adoption of AOSSL and/or HTTP Strict Transport Security (HSTS) as best practices to secure sensitive data between a user’s device and a web site. With the advent of widely available tools, criminals can "sidejack" cookies and data packets from unsuspecting users. Sidejacking allows hackers to intercept cookies (typically used to retain user-specific information such as username, password and session data) when they are transmitted without the protection of SSL encryption. Sites supporting AOSSL receive bonus points. This capability is assessed using the tools listed above to look for Strict Transport Security and is verified by auditors accessing the sites.
Malware, Malicious Links & Cross-Site Scripting – Sites will be scanned for malware and malicious links. Cross-site scripting will be assessed via public databases outlining reported vulnerabilities. Sites with vulnerabilities receive penalty points.
Bot and Botnet Protection – Sites will be checked for basic protection against web scraping, vulnerability scanning, scripted form completion, and other common bot-driven activities. Sites without basic bot protection will receive penalty points.
Web Application Firewall – Sites which have a web application firewall receive bonus points. Web Application Firewalls monitor HTTP conversations and block common attacks such as cross-site scripting (XSS) and SQL injections.
(New in 2017) Vulnerability Reporting Mechanisms - Recognizing the importance of having a vehicle for responsible reporting of site vulnerabilities, a search using common keywords will be conducted on audited sites and on third-party sites to look for the presence of a vulnerability reporting mechanism. Terms will include but are not limited to "bug reports," "bug bounty," "site vulnerabilities" and "vulnerability disclosures." Sites supporting a vulnerability reporting mechanism will receive bonus points. For more information visit the U.S Department of Commerce, NTIAs Cybersecurity Vulnerability Multi-stakeholder overview.
DNS & DDoS Resiliency – Sites will be analyzed for their ability to handle DNS and denial of service attacks. Sites with sufficient resilience will receive bonus points.
- Data sharing language
- Data retention language
- Data sharing with third parties
- Mention of adherence to COPPA
- Do Not Track (DNT) disclosure
Access to Previous Versions – Sites that allow access to previous versions of "marked-up" or "redlined" privacy policies will receive bonus points. While a date stamp or a page may inform a user the policy has changed, without access to previous version(s) the user will not know what has changed.
Privacy Policies with Icons – Building on layered notices, sites which use consumer friendly icons receive bonus points. See example of Publishers Clearing House.
Tag Management Systems or Privacy Solutions – Sites supporting multiple trackers often utilize tag management systems or privacy solutions to inventory and manage those trackers, since without such oversight sites often end up with old trackers that are active, but no longer have a business purpose. Sites supporting such systems receive bonus points.
Private WHOIS – To support transparency and allow consumers to see who owns a domain, WHOIS records of top sites should be public. Sites with a private WHOIS record receive penalty points.
FTC/FCC/State Settlements & Data Breaches – Organizations which have received a settlement or experienced a data breach since January 1, 2016 will receive penalty points.