Home2018 Online Trust Audit Methodology

2018 Online Trust Audit Methodology

The 2018 Online Trust Audit is the 10th year OTA will be conducting an independent analysis and benchmark report of the adoption of security standards and responsible privacy practices. This methodology reflects comments received in response to OTA's public call for comments in September 2017, commonly accepted best practices and emerging threat vectors.

The Audit will analyze more than 1,000 consumer-facing websites including top online retailers, banks, consumer service sites, government agencies, news and media companies, and Internet Service Providers, mobile carriers, email providers and web hosters. New in 2018 will be an audit of the healthcare sector, which will include the top hospital networks, pharmacies, health insurance companies and testing labs.

Sites are eligible to receive 300 total base points, including up to 100 points in each category. Bonus points are available for implementation of emerging best practices and penalties are assessed for vulnerabilities, breaches and regulatory settlements. The 2018 scoring has been expanded and enhanced with additional weight and granularity given to key practices. 

To qualify for the Honor Roll status, sites need to receive a composite score of 80% or better and a score of at least 60 in each of the three separate categories. Each sector will be scored in three categories:

The Audit is planned to be completed between late August and mid-September 2018. OTA announced the criteria in August 2018 via press release and on the OTA website and blog. It should be noted that this research is based on a “slice of time” and individual companies may have adopted or change their security and privacy practices after the Audit. OTA recognizes that the sites examined might be using other technologies (which our tools or research did not detect) to authenticate domains or subdomains, secure their infrastructures, track users on their sites, etc.

Due to the sensitivity of this data and risk of disclosing vulnerabilities, individual organization’s scores and data will not be publicly available. Information will be provided to site owners upon written request and verification. 


Email Authentication (SPF, DKIM & DMARC) – The report will analyze emails and the respective DNS infrastructure of leading sites and subdomains. Email authentication assesses efforts to protect users from domain and email spoofing via the adoption of two industry leading protocols – Sender Policy Framework (SPF) and Domain-Keys Identified Mail (DKIM). 

Sites receive a maximum of 100 points by 1) implementing both SPF and DKIM authentication at the top level domain (e.g., yourdomain.com) as well as on their respective subdomains (e.g., email.yourdomain.com), and 2) implementing a DMARC record with a “reject” policy at the top level domain. Partial credit is given for support of SPF, DKIM and DMARC at the subdomain level. Verification of DKIM-signed email requires review of the email headers of individual emails. OTA subscribes to email newsletters and/or submits inquiries to sites to review responses, which provides increased granularity of email data. 

Results are integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll. Invalid SPF and DMARC records do not receive credit. Likewise, "naked" DMARC records (a policy of "none" with no reporting) do not receive credit.  

Domain Locking – Domain locking is a security enhancement offered by most registrars to help prevent unauthorized transfers of your domain to another registrar or web host by locking your domain name servers. When your domain is locked, you'll be substantially protected from unauthorized third parties who might try to redirect your name servers or transfer your domain without your permission. Sites receive a penalty if their domain is not locked.

Transport Layered Security (TLS) for Email – Sites which implement "opportunistic TLS" will receive bonus points, which will be increased in 2018. TLS helps prevent eavesdropping on email as it moves between email servers that have enabled TLS protections for email. Just as TLS can be used to secure web communications (HTTPS), it can secure email transport. To maximize the content security and privacy, TLS is required between all the servers that handle the message including hops between internal and external servers. TLS adoption will be assessed using TLS databases provided by Twitter, Google and others as well as examination of email received from audited entities.

IPv6 & Domain Name System Security Extension (DNSSEC) – Testing will be completed using public tools and browser plug-ins. Sites adopting IPv6 and/or DNSSEC will receive bonus points.

Multi-Factor Authentication – Sites which provide an option for multi-factor authentication will receive bonus points.  By judiciously combining a strong password with additional factors, such as a fingerprint or a single use code delivered in a text message, accounts are better protected from account takeovers and unauthorized password resets.  


Server and TLS/SSL Configuration – Sites will be evaluated using a combination of data and tools which provide visibility into the server architecture, configuration and digital certificates. These tools check for weak keys, protocols, algorithms and server misconfigurations that can enable attackers to exploit system vulnerabilities and compromise TLS/SSL communications. Sites are also examined for application and network security, IP reputation and patching cadence. A blended score from the results of these tools plus presence of Always On SSL makes up the 100 baseline points in this category.

Always On SSL (AOSSL) – Sites are evaluated for the adoption of AOSSL, "HTTPs everywhere" and/or HTTP Strict Transport Security (HSTS) as best practices to secure sensitive data between a user’s device and a web site. With the advent of widely available tools, criminals can "sidejack" cookies and data packets from unsuspecting users. Sidejacking allows hackers to intercept cookies (typically used to retain user-specific information such as username, password and session data) when they are transmitted without the protection of TLS/SSL encryption. This capability is assessed using the tools listed above to look for Strict Transport Security and is verified by auditors accessing the sites. In 2018, AOSSL is part of the 100 baseline points (it earned bonus points in previous years).

Organization Validation (OV) and Domain Validated (DV) certificates receive neutral scoring. Organization Validation (OV) and Extended Validation (EV) certificates contain the verified name of the entity that controls the website. Certificate authorities (CAs) issuing these certificates check with third parties to establish the official name of the organization and where they are located. By contrast, Domain-Validated certificates are typically verified through automated processes. A DV certificate contains no identifying information in the organization name field. Typically, this value just re-states the domain name or simply says "Not Validated." Although the certificate supports transaction encryption, the end-user cannot confirm the identity of the organization on the other end.

Extended Validation SSL Certificates (EV SSL) – EV SSL offers visible confirmation of site identity to the user. Cybercriminals target business-to-business, social networking and government sector sites with non-EV Certificates. Acquiring an Extended Validation certificate requires extensive verification by the certificate authority. Sites with EV SSL Certificates receive bonus points.

Certificate Authority Authorization (CAA) – CAA is a security measure that allows domain owners to specify in their Domain Name Servers (DNS) which certificate authorities are authorized to issue certificates for that domain. Sites supporting CAA, as determined by the server configuration tools mentioned above, will receive bonus points.

Malware, Malicious Links & Cross-Site Scripting – Sites will be scanned for malware and malicious links. Cross-site scripting will be assessed via public databases outlining reported vulnerabilities. Sites with vulnerabilities receive penalty points.

Bot and Botnet Protection – Sites will be checked for basic protection against web scraping, vulnerability scanning, scripted form completion, and other common bot-driven activities. Sites without basic bot protection will receive penalty points.

Web Application Firewall – Sites which have a web application firewall receive bonus points. Web Application Firewalls monitor HTTP conversations and block common attacks such as cross-site scripting (XSS) and SQL injections.

Vulnerability Reporting Mechanisms – Recognizing the importance of having a vehicle for responsible reporting of site vulnerabilities, a search using common keywords will be conducted on audited sites and on third-party sites to look for the presence of a vulnerability reporting mechanism. Terms will include but are not limited to "bug reports," "bug bounty," "site vulnerabilities" and "vulnerability disclosures." Sites supporting a vulnerability reporting mechanism will receive bonus points. For more information visit the U.S Department of Commerce, NTIAs Cybersecurity Vulnerability Multi-stakeholder overview and see this CSO article.


Privacy Policy & Tracking Score – Sites will be analyzed for their privacy policy and data collection practices (55 points) as well as the privacy practices of third-party trackers on the site (45 points). The privacy policy score evaluates privacy risk based on a website's published policies about protection of personal data and the privacy qualifications of third-parties seen to be collecting data on the site. Website privacy policies regarding sharing, deletion, retention policies, disclosure notices and vendor confidentiality are reviewed by analysts.

Scoring for third-party tracking companies (reflecting policies on anonymity, boundaries, choice, retention and oversight) are weighted based on their prevalence in site scans. It is important to note that scores are dynamic and can change based on the mix of third-party tracking and revisions to privacy policies. Results are integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll. 

Components included in the core privacy policy score include:

  • Privacy policy link discoverable on home page
  • Data sharing language
  • Data retention language
  • Data sharing with third parties
  • Layered notices 
  • Mention of adherence to COPPA
  • Do Not Track (DNT) disclosure 
  • Date stamp at the top of the privacy policy 
  • Access to previous versions (previously awarded bonus points, part of baseline in 2018)

Privacy Policies with Icons – Building on layered notices, sites which use consumer friendly icons receive bonus points. See example of American Greetings

Do Not Track Browser Settings (DNT) – A DNT signal asserts a user's request to not collect and share their online data. In response to the State of California disclosure requirement for a site regarding Do Not Track, sites’ privacy policies will be evaluated for compliance. Disclosure of the DNT policy is part of the baseline scoring, and sites which publicly disclose they are honoring the browser-based DNT setting receive bonus points. Such an assertion would be in addition to any such notice a user is presented when visiting a site and does not preempt any such notice. For additional information see the updated California Guidelines.

Multi-Lingual Policy – Offering the privacy policy in multiple languages provides critical information to a broader audience. Sites offering their privacy policy in multiple languages receive bonus points.

Tag Management Systems or Privacy Solutions – Sites supporting multiple trackers often utilize tag management systems or privacy solutions to inventory and manage those trackers, since without such oversight sites often end up with old trackers that are active, but no longer have a business purpose. Sites supporting such systems receive bonus points.

Private WHOIS – To support transparency and allow consumers to see who owns a domain, WHOIS records of top sites should be public. Sites with a private WHOIS record receive penalty points.

FTC/FCC/State Settlements & Data Breaches – Organizations which have received a settlement or experienced a data breach since January 1, 2017 will receive penalty points.

General Data Protection Regulation (GDPR) Language – GDPR went into effect on May 25, 2018 and sets requirements for handling of data on sites that support EU citizens and residents. It specifies several elements that must be addressed in privacy statements, including an easy to read policy, disclosure of what data is collected and for what reason (including cookies), disclosure of specific third parties who receive data, identification of a Data Protection Officer (DPO), how sensitive personal data is handled and support for users' requests to access data. New for 2018, these elements will be tracked as part of the privacy policy assessment. If the majority of these elements are incorporated into the privacy policy, bonus points will be awarded.