About Us Membership Resources OTA Members Login
2013 HONOR ROLL - METHODOLOGY

Press Release          Report Downloads           Honor Roll Recipients           Segment Data          Methodology

2013 Honor Roll

The 2013 Honor Roll included a composite analysis of a site's security and privacy practices focusing on three major categories, including thirteen criteria as outlined below.  Sites were eligible to receive 300 points combined in three categories as well as bonus points for adoption of leading edge practices.  The categories include: 1) Domain, Brand & Consumer Protection, Site, Server & Infrastructure Security and Data Protection & Privacy.

The 2013 report expands the criteria and provides additional weight and granularity  to key practices.  Sites which receive a composite score of 80% or better AND score at least 55 points in each of the three categories, qualified for the 2013 Honor Roll.  Data sampling of sites, their DNS and email and privacy policies were completed between April 21 and May 20th.  In total over 500 million emails were examined and approximately 10,000 web pages reviewed.

History & TimeLine of the OTA Honor Roll - A recap since the inception of OTA in August 2004.

Domain, Brand & Consumer Protection

Email Authentication (SPF & DKIM)
The report analyzed over 500 million emails and the DNS infrastructure of leading sites. It assessed efforts to protect users from domain and email spoofing as spear phishing via the adoption of SPF and DKIM.  Sites received maximum scores by implementing both SPF and DKIM as well as implementing authentication at the top level domain (i.e. yourdomain.com).  Verification of DKIM signed mail required evidence of email being signed and was conducted by OTA data partners.  Results were integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll.  Verification of SPF records was completed using the OTA tool, https://otalliance.org/resources/authentication/spflookup.html

Domain-based Message Authentication, Reporting & Conformance (DMARC)
DMARC standardizes how email receivers perform email authentication using the SPF and DKIM mechanisms.  Sites that have published DMARC records receive a positive score with additional scoring for sites that have published reject or quarantine instructions.  Verification was completed using the OTA DNS test record query tool, https://otalliance.org/resources/authentication/spflookup.html.  This criteria was introduced as a bonus score in 2012 and moved to a core part of the composite scoring representing broad adoption by mailers and the ISP community.

Domain Locking - Domain locking is a security enhancement offered by most registrars to help prevent unauthorized transfers of your domain to another registrar or web host by locking your domain name servers. When your domain is locked, you'll be substantially protected from unauthorized third parties who might try to misdirect your name servers or transfer your domain without your permission.  Sites receive negative points if their domain is not locked.  More Information

Site, Server & Infrastructure Security

SSL Server Configuration - Sites were evaluated using a combination of  data from Qualys SSL Labs and analysis using the High-Tech Bridge SA, ImmuniWeb® SSL Certificate Monitor service.  The results provides several advantages over simply checking for EV SSL as was done in past years.  Not only do we check for weak keys, protocols, and algorithms, but we now identifies server misconfigurations that can enable attackers to exploit system vulnerabilities and compromise SSL communications.  Updates to 2013 SSL Labs Security Requirements

Extended Validation SSL Certificates (EV SSL)  -  EV SSL offers trust mechanisms confirming the identity of the site to the user.  The 2012 analysis focused on all sites which have SSL connections, no longer limiting the evaluation to only consumer facing e-commerce or banking sites.  This change is the result of the increased targeting of business-to-business, social networking, and government sector sites for user data. Site who have implemented EV SSL Certificates receive bonus points.  

Always On SSL (AOSSL) - AOSSL is a best practice to secure sensitive data, especially for users of public Wi-Fi hot spots.  With the advent of widely available tools, criminals can "sidejack" cookies and data packets from unsuspecting users.  Sidejacking allows hackers to intercept cookies (typically used to retain user-specific information such as username, password and session data) when they are transmitted without the protection of SSL encryption. Sites supporting AOSSL receive additional points.  This capability was measured using the Qualys SSL Server Test to look for Strict Transport Security. 

2048 bit key or Elliptic Curve Cryptography (ECC) Certificates - Sites who have adopted 2048-bit certificates of ECC, receive bonus points.  ECC adds security to SSL Certificates, offering a secure web experience that is favorable to users and can reduce server and virtual server overhead needs for processing connections.  ECC supports the National Institute of Standards and Technology (NIST) requirement to migrate from RSA 1024-bit crypto to 2048-bit certificates by January 2014.  More Information

Domain Name System Security Extension (DNSSEC) - Testing for DNSSEC was completed using a custom-built tool from Internet Identity (IID) that queried DNSSEC records via "dig" requests.  Accounting for the risk of DNS errors, the analysis was run twice during the test period.  Sites adopting DNSSEC receive bonus points.

Data Protection, Privacy & Transparency

Privacy Policy  -  Using third-party provided data from PrivacyChoice via their Privacyscore research and additional OTA criteria, sites were analyzed for their privacy policy and data collection practices.  A Privacyscore evaluates privacy risk based on a website's published policies about protection of personal data, and the privacy qualifications of third-parties seen to be collecting data on the site.  Website privacy policies as to sharing, deletion, disclosure notices and vendor confidentiality were reviewed by analysts during the period between April 25 and May 20, 2013.  Privacyscores for third-party tracking companies (reflecting policies on anonymity, boundaries, choice, retention and oversight) were weighted based on their prevalence in site scans.  Privacyscores change over time based on the mix of third-party tracking and revisions to privacy policies.  Results from Privacyscore were integrated into the composite scoring and factored as a component of the baseline points required to qualify for the Honor Roll.  For a review of the Privacyscore methodology, visit http://www.privacyscore.com/faq.

Third Party Tracking on Site (see above)

Honoring of Do Not Track Browser Settings (DNT) - Websites who publically disclose the status of honoring or not honoring the browser based DNT setting receive bonus points.  Such an assertion would be in addition to any such notice a user is presented when visiting a site and does not preempt any such notice.  A DNT signal asserts a user's request to not collect and share their online data will receive bonus points. Sites with no assertion supporting or ignoring the DNT signal composite score will not be impacted.  As the standard is evolving with the W3C, it is recognized many sites are reviewing their position.  As proposed support of DNT by a site is voluntary, but draft legislation has been proposed to require sites to honor the preference. More Information

Public vs. Private WHOIS registration - Sites that are registered by proxy or private registration received a negative score, reflecting a lack of transparency. While it is recognized that sites may choose to opt-in for private domain name registration, public facing sites are discouraged from doing so and consumers should exercise caution when interacting with sites that have made their domain information totally private.  Results were integrated into the composite scoring as a negative score for sites with private registrations and factored as a component of the baseline points required to qualify for the Honor Roll.  More Information

Data Breach & Loss Incidents - Companies who have experiences a data breach or a data loss incident since April 2011, received negative points.

FTC / State Settlements - Companies which have been in violation of the FTC Act including settlements and judgments since April 2011, focusing on consumer protection, including but not limited to deceptive advertising, privacy and data security practices receive negative points. See http://business.ftc.gov/legal-resources/8/

Updated 5/16/2013