HomeInitiatives2017 Online Trust Audit Methodology

2017 Online Trust Audit Methodology

2017 Online Trust Audit Methodology - Draft

Online Trust Audit & Honor Roll

Briefing Deck (Updated March 6, 2017)

The 2017 Online Trust Audit is the 9th year OTA will be conducting an independent analysis and benchmark report of the adoption of security standards and responsible privacy practices.   This methodology reflects comments received in response to OTA's public call for comments in September 2016, commonly accepted best practices and emerging threat vectors.

The 2017 Audit will evaluate approximately 1,000 websites across multiple sectors including; the Internet Retailer Top 500, the FDIC Top 100 Banks, the Top 100 Consumer sites, the Top 50 Federal Government sites, the Top 100 News/ Media sites and OTA Members.  New for 2017 is the addition of a new sector including the top 50 ISPs, Carriers and Mail box providers.

Sites are eligible to receive 300 total base points, including up to 100 points in each category. Bonus points are available for implementation of emerging best practices and penalties are assessed for vulnerabilities, breaches and regulatory settlements.  The 2017 scoring has been expanded and enhanced with additional weight and granularity given to key practices. 

To qualify for the Honor Roll status, sites need to receive a composite score of 80% or better and a score of at least 55 in each of the three separate categories. Each sector will be scored in three categories:

The Audit is planned to be completed between mid-April and the end of May, 2017. In total, it is estimated that more than 500 million email headers and approximately 100,000 web pages will be analyzed. It is important to note a site's policies and practices may change after the sampling and the data only reflects findings based on this period of time. With the goal to drive adoption and awareness of best practices allowing all companies the ability to access their status and optimize their scores, OTA announced the criteria in late January 2017 via press release and on the OTA website, blog and external facing newsletters.

It should be noted that this research is based on a “slice of time” and individual companies may have adopted or change their security and privacy practices after the Audit. OTA recognizes that the sites examined might be using other technologies (which our tools or research did not detect) to authenticate domains or subdomains, secure their infrastructures, track users on their sites, etc. Due to the sensitivity of this data and risk of disclosing vulnerabilities, individual organization’s scores and data will note be publicly available. Information will be provided to site owners upon written request and verification. For details, including reporting fees, please send an email to admin @ otalliance.org.

DOMAIN, BRAND & CONSUMER PROTECTION

Email Authentication (SPF, DKIM & DMARC) – The report will analyze more than 500 million emails and the respective DNS infrastructure of leading sites and subdomains. Email authentication assesses efforts to protect users from domain and email spoofing via the adoption of two industry leading protocols – Sender Policy Framework (SPF) and Domain-Keys Identified Mail (DKIM). Sites receive a maximum of 100 points by 1) implementing both SPF and DKIM authentication at the top level domain (e.g., yourdomain.com) as well as on their respective subdomains (e.g., email.youremail.com), and 2) implementing a DMARC record with a “reject” policy. Verification of DKIM-signed email requires review of the email headers of individual emails via data sampling providing by Agari, Microsoft, ValiMail and other data providers Augmenting previous years’ methodology, OTA subscribes to email newsletters and/or submits inquiries to sites to review responses, which provides increased granularity of email data. Results are integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll. Verification of SPF and DMARC records will be completed using the OTA DNS record lookup tool.  

Domain Locking – Domain locking is a security enhancement offered by most registrars to help prevent unauthorized transfers of your domain to another registrar or web host by locking your domain name servers. When your domain is locked, you'll be substantially protected from unauthorized third parties who might try to redirect your name servers or transfer your domain without your permission. Sites receive a penalty if their domain is not locked.

Transport Layered Security (TLS) for Email – Sites which implement "opportunistic TLS" will receive bonus points. TLS helps prevent eavesdropping on email as it is carried between email servers that have enabled TLS protections for email. Just as TLS can be used to secure web communications (HTTPS), it can secure email transport. To maximize the content security and privacy, TLS is required between all the servers that handle the message including hops between internal and external servers. TLS adoption will be assessed using TLS databases provided by Twitter, Google and others as well as examination of email received from audited entities.

IPv6 & Domain Name System Security Extension (DNSSEC) – Testing will be completed using public tools and browser plug-ins, including data provided by Verisign and Infoblox. Sites adopting IPv6 and/or DNSSEC will receive bonus points.

SITE SECURITY & RESILIENCY

Server and SSL Configuration – Sites will be evaluated using a combination of data and tools from DigiCert, GlobalSign, Hardenize, High-Tech Bridge SA, Qualys Labs, Security Scorecard, SiteLock and Symantec. These tools provide visibility into the server architecture, configuration and digital certificates. Testing checks for weak keys, protocols, algorithms, and server misconfigurations that can enable attackers to exploit system vulnerabilities and compromise SSL communications. A blended score from the results of these tools makes up the vast majority of the 100 baseline points in this category. New in 2017, baseline scoring will also include assessment of whether a site has a vulnerability reporting mechanism discoverable from the home page (via link or search). Responsible and coordinated vulnerability disclosure reporting is widely recognized as a best practice and security fundamental for the U.S. Department of Commerce, NTIA and the FTC.

Organization Validation (OV) certificates receive neutral scoring, while sites with Domain Validated (DV) and self-signed certificates receive a penalty. Organization Validation (OV) and Extended Validation (EV) practices contain the verified name of the entity that controls the website. Certificate authorities (CAs) issuing these certificates check with third parties to establish the official name of the organization and where they are located. By contrast, Domain-Validated certificates are typically verified through automated processes. A DV certificate contains no identifying information in the organization name field. Typically, this value just re-states the domain name or simply says "Not Validated." Although the certificate supports transaction encryption, the end-user cannot trust the certificate to confirm who is on the other end.

Extended Validation SSL Certificates (EV SSL) – EV SSL offers visible confirmation of site identity to the user. The 2017 analysis will focus on all sites with SSL connections, not limiting the evaluation to consumer facing e-commerce or banking sites. Cybercriminals target business-to-business, social networking and government sector sites with non-EV Certificates. Acquiring an Extended Validation certificate requires extensive verification by the certificate authority. Sites with EV SSL Certificates receive bonus points.

Always On SSL (AOSSL) – Sites are evaluated for the adoption of AOSSL and/or HTTP Strict Transport Security (HSTS) as best practices to secure sensitive data between a user’s device and a web site. With the advent of widely available tools, criminals can "sidejack" cookies and data packets from unsuspecting users. Sidejacking allows hackers to intercept cookies (typically used to retain user-specific information such as username, password and session data) when they are transmitted without the protection of SSL encryption. Sites supporting AOSSL receive bonus points. This capability is assessed using the tools listed above to look for Strict Transport Security and is verified by auditors accessing the sites.

Malware, Malicious Links & Cross-Site Scripting – Sites will be scanned for malware, malicious links and susceptibility to cross-site scripting. Sites with vulnerabilities receive penalty points.

Bot and Botnet Protection – Sites will be checked for basic protection against web scraping, vulnerability scanning, scripted form completion, and other common bot-driven activities. Sites which have such protection will be awarded bonus points.

Web Application Firewall – Sites which have a web application firewall receive bonus points. Web Application Firewalls monitor HTTP conversations and block common attacks such as cross-site scripting (XSS) and SQL injections.

Malvertising – Cybercriminals have recognized the security vulnerability of the advertising ecosystem and are increasingly distributing ads with malicious payloads and code in an effort to compromise users’ devices and business systems. Known as malicious advertising, or “malvertising,” it poses a growing threat to everyone who accesses ad supported content online, as well as to ad supported services. Sites which have known malvertising incidents receive penalty points.

(New in 2017) Multi-Factor Authentication – Sites which provide an option for multi-factor authentication will receive bonus points.  By judiciously combining a strong password with additional factors, such as a fingerprint or a single use code delivered in a text message, accounts are better protected from account takeovers and unauthorized password resets.  

(New in 2017) Vulnerability Reporting Mechanisms - Recognizing the importance of having a vehicle for responsible reporting of site vulnerabilities, site's home pages and search functions will be searched using common key words.  Terms will include but are not limited to bug reports, bug bounty, site vulnerabilities and vulnerability disclosures. For more information visit the U.S Department of Commerce, NTIAs Cybersecurity Vulnerability Multi-stakeholder overview.

DNS & DDoS Resiliency – Sites will be analyzed for their ability to handle DNS and denial of service attacks. Sites with sufficient resilience will receive bonus points.

DATA PROTECTION, PRIVACY & TRANSPARENCY

Privacy Policy & Tracking Score – Sites will be analyzed for their privacy policy and data collection practices (50 points) as well as the privacy practices of third-party trackers on the site (50 points). The privacy policy score evaluates privacy risk based on a website's published policies about protection of personal data and the privacy qualifications of third-parties seen to be collecting data on the site. Website privacy policies regarding sharing, deletion, retention policies, disclosure notices and vendor confidentiality are reviewed by analysts. Scoring for third-party tracking companies (reflecting policies on anonymity, boundaries, choice, retention and oversight) are weighted based on their prevalence in site scans. Sites are crawled for a period of at least 96 hours to observe data collection and onsite tracking. It is important to note that scores are dynamic and can change based on the mix of third-party tracking and revisions to privacy policies. Results are integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll. 

Components included in the core privacy policy score include:

  • Privacy policy link idscoverable on home page
  • Data sharing language
  • Data retention language
  • Data sharing with third parties
  • Layered notices (OTA's Privacy Policy as an example)
  • Mention of adherence to COPPA
  • Date stamp at the top of the privacy policy (previously awarded bonus points, now part of baseline)

Access to Previous Versions – Sites that allow access to previous versions of "marked-up" or "red-lined" privacy policies will receive bonus points.  While a date stamp or a page may inform a user that the policy has changed, but without access to previous version(s) the user will not know what has changed.

Privacy Policies with Icons – Building on layered notices, sites which use consumer friendly icons receive bonus points. See example of Publishers Clearing House 

Do Not Track Browser Settings (DNT) – A DNT signal asserts a user's request to not collect and share their online data. In response to the State of California disclosure requirement for a site regarding Do Not Track, sites’ privacy policies will be evaluated for compliance. Sites that do not disclose their DNT policy receive a penalty (new in 2017 – used to be part of baseline points), and sites which publicly disclose they are honoring the browser-based DNT setting receive bonus points. Such an assertion would be in addition to any such notice a user is presented when visiting a site and does not preempt any such notice. For an example of a DNT notification best practice review the OTA Privacy Policy. For additional information see the updated California Guidelines