An overview of the concept and use of domain-name entropy

Introduction

In this article, I present an overview of a series of 'proof-of-concept' studies looking at the application of domain-name entropy as a means of clustering together related domain registrations, and serving as an input into potential metrics to determine the likely level of threat which may be posed by a domain.

In our previous studies, we utilised the mathematical concept of Shannon entropy^[1], providing a measure of the amount of information stored in a string of characters (or, equivalently, the number of bits required to optimally encode the string). The idea was applied to the second-level domain name (SLD) part of each domain (i.e. the portion of the domain name before the dot - such as 'google' in 'google.com'), and broadly means that short domain names, or those with large numbers of repeated characters, will have low entropy values, whereas longer domain names, or those with large numbers of distinct characters, will have higher entropy.

The background to this analysis is the fact that domains registered for egregious purposes (such as spamming, malware distribution, or botnet creation) may be more likely to be registered in bulk by bad actors using automated algorithms^[2], which typically results in the generation of long, non-sensical (i.e. high entropy) domain names, which have the added benefit of not containing brand-related keywords and are typically therefore harder to detect using classic brand-monitoring techniques. The idea is that domains registered by a particular infringer for a specific campaign are likely all to be generated using the same algorithm, and may therefore have similar or identical entropy values.

Overview of previous studies

In our initial proof of concept^[3], we considered the set of all domains registered on a particular day - a sample of around 205,000 domains. The advantage also of considering a set of domains with a common registration date is that it presents the possibility for one or more groups of automated bulk registrations (which are typically all registered at the same time) to be present.

Within the dataset, a range of domain entropy values was present, from a minimum of 0.000, to a maximum of 4.700, and with 92.3% of the dataset having values below 3.500. (see Figure 1). The top 1,000 highest-entropy domains (i.e. the top 0.49%) had entropy values in excess of 3.823, and accounted for the majority of examples which appeared visually to feature 'random' SLD strings. Within this high-entropy subset, a number of additional characteristics were indicative that many may have been registered for nefarious purposes, including the prominence of use of consumer-grade registrars and privacy-protection services, and the extent of the presence of active MX records amongst these new registrations (in 27.5% of the cases - indicating that these domains have been configured to be able to send and receive e-mails and therefore could potentially be associated with phishing activity).

Figure 1: Cumulative proportion of domains with entropy less than the value shown on the horizontal axis, from the dataset in the initial proof-of-concept study

Indeed, at least one apparent 'cluster' of suspicious registrations was found to be present within the dataset, comprising a group of 125 .buzz ('dot-buzz') domains, all with an identical high entropy value (3.907), registered via a common registrar and associated with groups of similar IP addresses. At the time of analysis, many of the domains registered to Chinese-language, gambling-related websites, likely representing either an affiliate revenue generation scheme, or 'dummy' content serving to 'mask' higher-threat content which may only have been visible in specific geographic regions, or which may have been planned for subsequent upload.

In a follow-up study^[4], I considered a month's worth of registrations of domains with names containing any of the top ten most valuable brands in 2022. Similarly, the high entropy domain names within this dataset included groups of apparently related, coordinated 'clusters' of domains, several of which appeared intended for fraudulent use and were consistent with registration via automated generation algorithms. For example, seven of the top eight domains in the dataset (by entropy values) had similar names of the form 'google-site-verificationXXXXXX.com' (or .net) (where 'XXXXXX' was a long string of apparently random characters), and a series of groups of 'microsoft' examples was identified, including keywords such as 'cloudworkflow', 'netsuites' and 'cloudroam'.

Comparison with other work

Other studies taking similar approaches to the analysis of domain entropy also reach similar conclusions. For example, an analysis outlined in a blog posting by Tiberium^[5] states that the use of an entropy threshold of >3.1 (as an indicator of potential concern) correctly classifies 80% of NCSC malicious domains, and incorrectly classifies only 8% of the top 1000 most popular (legitimate!) domains overall (cf. Table 1).

Domain name	Entropy value
google.com	1.918
youtube.com	2.522
facebook.com	2.750
twitter.com	2.128
instagram.com	2.948
baidu.com	2.322
wikipedia.org	2.642
yandex.ru	2.585
yahoo.com	1.922
whatsapp.com	2.500

Table 1: Entropy values of the SLDs of the top ten most popular websites according to Similarweb^[6]

Additionally, an article published by Splunk^[7] looking at the entropy values of fully qualified domain names, i.e. also including subdomain names - also states that high-entropy examples are consistent with the use of domain generation algorithms, and may be indicative of association with malware (e.g. in 'beaconing') and other web exploits. Comparable approaches and conclusions can also be found in a range of other studies^[8,9,10], with some finding improvements in the reliability of threat determination through the use of alternative measures such as relative entropy (essentially, a comparison against the character distribution observed in a dataset of known legitimate domains, so as to provide a better measure of the randomness arising from automated algorithmic registrations)^[11].

Conclusions

Domain-name entropy analysis has applications in at least two key areas of brand protection. The first of these is the ability to 'cluster' together related infringements, which has a number of benefits, including the ability to identify serial infringers and instances of bad-faith activity, for targeted and effective bulk enforcement actions. The second key area is as an input into algorithms to quantify the likely level of threat which may be posed by an online feature such as a new domain registration. Threat determination is essential in allowing prioritisation of results for analysis, enforcement, or content-change tracking.

All other factors being equal, there is some indication that high-threat domains - particularly those associated with automated registrations by domain-name generation algorithms - may have a tendency to sit at the higher-entropy end of the spectrum (and, furthermore, that domain names generated using a particular algorithm may be likely to have similar entropy values). This statement runs alongside the assertion that legitimate domains may (in general) be more likely to have lower entropy values, particularly where there is a desire for legitimate businesses to utilise strongly branded, short, memorable web addresses - as can be seen in many of the globally most popular websites.

References

[1] https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf

[2] https://interisle.net/sub/CriminalDomainAbuse.pdf

[3] https://www.linkedin.com/pulse/investigating-use-domain-name-entropy-clustering-results-barnett/

[4] https://www.linkedin.com/pulse/entropy-analysis-registered-domain-names-relating-top-david-barnett/

[5] https://www.tiberium.io/blog/chapter-2-classifying-domains-through-string-entropy/

[6] https://www.similarweb.com/top-websites/

[7] https://www.splunk.com/en_us/blog/security/random-words-on-entropy-and-dns.html

[8] https://hurricanelabs.com/blog/dns-entropy-hunting-and-you/

[9] https://www.logpoint.com/en/blog/embracing-randomness-to-detect-threats-through-entropy/

[10] https://suleman-qutb.medium.com/use-of-shannon-entropy-estimation-for-dga-detection-9ded275795ca

[11] https://redcanary.com/blog/threat-hunting-entropy/

This article was first published on 3 July 2023 at:

https://circleid.com/posts/20230703-an-overview-of-the-concept-and-use-of-domain-name-entropy

The 'Millennium Problems' in Brand Protection

As the brand protection industry approaches a quarter of a century in age, following the founding of pioneers Envisional^[1] and MarkMonitor^[2] in 1999, I present an overview of some of the main outstanding issues which are frequently unaddressed or are generally only partially solved by brand protection service providers. I term these the 'Millennium Problems' in reference to the set of unsolved mathematical problems published in 2000 by the Clay Mathematics Institute^[3], and for which significant prizes were offered for solutions. Like their mathematical counterparts, the unsolved problems in brand protection will present significant benefits for any service providers able to develop and offer comprehensive solutions.

Brand protection basics

In their most basic sense, brand protection solutions generally consist of two components: monitoring (or, strictly, detection) of brand-related content on the Internet, and enforcement action to achieve the removal of infringing material. Monitoring is most usually carried out using technological solutions intended to identify relevant material on the Internet, across a range of relevant channels, typically using a combination of methodologies, namely: (i) Internet metasearching (i.e. the submission of relevant query terms to search engines) and web crawling; (ii) analysis of domain-name zone files (see Problem 2), to identify domains with names including brand-related terms (or variants); (iii) direct monitoring / searching on known sites of interest (see Problem 1); and (iv) other techniques, such as the use of spam traps and webserver logs, as used in phishing detection technologies^[4]. Many service providers will also make use of automated analysis tools, which can inspect the content of the identified webpages, and categorise and prioritise these results accordingly.

The 'Millennium Problems'

1. Social media monitoring

Whilst monitoring of content across social media platforms is a well-established element of many brand-protection service providers' product suites, it frequently remains extremely difficult to achieve anything approaching a comprehensive level of coverage. There are a number of reasons why this is the case. In general, social media content is most usually addressed using the 'direct site searching' approach (that is, using the search functionality typically in-built to the platforms themselves as a means of returning results), though some providers also have access to direct data feeds from the platforms (e.g. through an API). In general, a variety of types of content may be of interest, including brand references in usernames (e.g. associated with fake profiles), and the content of postings (e.g. associated with fraud, the sale of counterfeits, the spread of malware, brand disparagement, etc.) and elsewhere (including imagery, sponsored advertisements, and so on).

The main difficulty with the 'direct search' approach is that results presented to a user are often limited (sometimes significantly) unless the user is logged in to the social media platform. This can be circumvented by configuring a brand-protection monitoring tool to present itself to the platform as if it is a real user (with a registered account, handle (username) and password), or simply through the use of manual searches. Both of these approaches typically require the use of 'dummy' accounts and may be in contravention of the terms and conditions of the platforms themselves.

Other technological issues may also be problematic. Many social media platforms return results on an 'infinite scroll' basis (where additional results are continually added to the webpage as the user continues to scroll down through them), often with no indication of the total numbers of results which may be present, and many platforms also have specific access requirements, such as functionality only to be accessed via a mobile app (see Problem 7). Similarly, monitoring can be further complicated by sites where content is protected via the requirement to enter a CAPTCHA code, for example. It is also typically the case that the exact results returned to a user will be highly personalised, and dependent on their browsing history, interests, location, and personal demographic.

Some of these issues can be addressed through the development of partner relationships by brand-protection service providers with the platforms themselves. However, even in cases where the platforms are amenable to this approach, some of the above technological issues may remain difficult to address.

2. Comprehensive ccTLD monitoring

Another of the core elements of many brand protection service offerings is often a domain monitoring capability; that is, the ability to identify domains whose names include the name of the brand being infringed (and/or other relevant keywords). As a special subset of general Internet content, branded domain names are often of particular interest by virtue of their greater visibility (e.g. higher ranking in search-engine results) and the more explicit nature of the IP abuse (and an associated greater range of enforcement options)^[5]. Branded domain names have been noted in many previous studies as being popular with bad actors in the creation of infringing content of a variety of types, including phishing sites^[6], sites offering the sale of counterfeits, and sites claiming false affiliation or including disparaging content.

The primary source of data for domain monitoring is usually the analysis of zone files, which are data files published by the registry organisations responsible for overseeing the infrastructure of each individual TLD (top-level domain, or domain extension - such as .com), and which contain a list of all existing registered domains across that extension. By comparing the content of a zone file with that from the previous day, it is possible to identify new domain registrations (as well as dropped, or lapsed, domains) and filter this list for those examples containing a brand name or keyword of interest. Domain monitoring solutions can (and, in general, should) also make use of zone-file analysis to allow identification of the full pre-existing 'landscape' of registered domain names of interest, across the TLDs in question, at the commencement of monitoring (so-called 'baseline' analysis). The most sophisticated domain monitoring solutions can also automatically check for variations of the brand strings (such as typos), which are frequently used by infringers to construct deliberately deceptive domain names^[7,8].

Zone files are generally available for most gTLDs (generic, or global, TLDs such as .com, .net, etc.) plus the new-gTLDs which have been launched in the period since 2012^[9], but are often not published (or may not be comprehensive) by the registry organisations responsible for other TLDs, particularly the country-specific examples (ccTLDs). For this reason, detection of relevant domains across ccTLD extensions is typically incomplete, and a number of techniques may typically be used in order to fill in the gaps. These might include parallel look-ups (checks for domains with the same second-level domain name - i.e. the part of the domain name to the left of the dot - as examples identified through zone-file analysis), exact-match queries (regular searches for the existence of domains with second-level domain name strings of particular relevance, such as a brand name), and Internet metasearching. However, each of these approaches has its own limitations and, even when all taken together, there can always be domain names of potential concern which are not detected through any of these methods. The next generation of domain monitoring solutions will need to better address these shortcomings, potentially involving factors such as the use of improved algorithms to 'guess' candidate domain names for checking, and/or the use of more comprehensive indexes of Internet content. Additionally, the building of specific relationships with country registries - potentially combined with regulatory changes regarding the availability of zone files - may also be relevant.

3. Third-party subdomain monitoring

The subdomain is the section of a URL prior to the domain name, from which it is separated by a dot (e.g. 'translate' in 'translate.google.com'). The owner of a domain name can create whatever subdomains they wish, and can point these URLs to associated web content (via the configuration of DNS settings). Accordingly, subdomains can be used to create brand-related URLs, and can be associated with many of the same types of infringements as domain names themselves^[10]. Subdomain-based abuse can also be particularly attractive to infringers, both because it avoids the requirement to register a brand-specific domain name^[11] (which bad actors know can easily be detected by brand owners employing domain-monitoring services) and because there can be a low cost associated with the creation of the URL, particularly where a service provider allowing the free registration of personalised subdomains (such as blogspot.com) is used.

Consequently, the ability to monitor generally for brand references in the subdomain name of arbitrary URLs can be of great value. Note that this is distinct from the (relatively much simpler) problem of monitoring the existence and content of subdomains of official domains under the ownership of the brand owner 'internal' subdomain monitoring), since all of the relevant information is contained in the DNS configuration files held by the brand owner's domain-name management service provider.

Conversely, the identification of brand-related subdomains on third-party ('external') domain names is much more difficult. In many cases, this is achieved purely using Internet metasearching techniques (i.e. finding only content which is indexed by search engines in response to brand-specific query terms). Whilst this does mimic the search techniques used by general Internet users (and thereby identify the 'highest-visibility' content), it will in general not find all potentially threatening content (e.g. URLs to which traffic is driven through other means, such as links in spam e-mails). This problem can be mediated to some degree through the use of other techniques, such as passive DNS analysis or certificate transparency (CT) analysis, or via explicit queries for the existence of specific subdomain names of interest. However, these techniques require prior identification of the specific domains to be monitored; generalised identification of brand-related subdomains remains a much harder problem to solve.

4. Circumventing site blocking and geoblocking

Site blocking and geoblocking are two long-established problems in brand monitoring. The former arises when a monitored site becomes aware of repeated search queries from a particular source, and restricts access to the site from the IP address in question. A site owner may choose to do this for a number of reasons, including protection of website performance (e.g. in preventing DDoS attacks), or for compliance with their own terms and conditions (e.g. where they state that information is not to be collected for commercial purposes, such as by brand-protection service providers). Geoblocking (or geotargeting) is a related issue, whereby the visible content of a website may vary depending on the geographical location of the visitor. Again, this may be implemented by a site owner for a range of reasons, including the tailoring of content to a local audience, search-engine optimisation, security, or legal compliance^[12]. However, geoblocking can also be employed by infringers as a means of evading detection, and can also present difficulties in enforcement, where it may be necessary to demonstrate exactly what content is visible from a specific remote location.

The solutions to these issues, from a brand-protection point of view, are relatively simple in principle, generally involving the use of proxies (standalone external machines serving as intermediate 'hops' through which search queries from a brand-protection service provider are routed, so as to 'mask' the originating IP address) in a range of remote locations, and/or (particularly for site blocking) the building of relationships with the sites being monitored, so that the monitoring service provider can gain permission for collecting the data. However, in practice this requires a great deal of investment in building the required infrastructure (such as hosting and maintaining the necessary proxies, and configuring the monitoring software to communicate with them) and establishing the necessary relationships. Furthermore, the construction of appropriate user interfaces to visualise and interpret the relevant information (such as the ability to compare the content of a particular website across a range of different user (i.e. proxy) locations, in cases where geoblocking or geotargeting may be an issue) can also be a complex prospect.

5. Clustering and open-source intelligence analysis

The subject areas of clustering and open-source intelligence (OSINT) are generally of greatest relevance for entity investigations, i.e. the process of using Internet searches to build a portfolio of information relating to an identified individual or website of interest. Such information can be used for a range of purposes, including background for on-the-ground investigations or goods seizures, or for legal cases, but can also be useful background for enforcement actions (e.g. in identifying clusters of related infringements for efficient bulk takedowns in a single action).

A number of technological solutions exist for visualising the links behind related entities, on the basis of common shared characteristics (such as e-mail addresses, telephone numbers, web-hosting information such as IP addresses, and so on) - i.e. 'clustering', but it is often the case that the characteristics themselves require identification through manual analysis processes. A great deal of additional efficiency can be built into the process, however, through the use of monitoring and analysis tools which can identify and extract this information automatically. This is relatively more straightforward in cases where the data can be extracted in a consistent manner (e.g. performing an IP-address look-up for any identified website of interest), and/or where the information is contained in a known location on a webpage with a fixed, pre-defined format (the 'contact details' section of a social-media profile page), such that a web scraper can be configured to pull out the content. It is a considerably more difficult enterprise to extract such information from general webpages where the structure of each page is not known in advance. In these cases, the approach generally needs to be based on the configuration of monitoring tools which are able to extract text-strings with the general format of (say) an e-mail address or telephone number. This then typically requires an element of post-processing to 'clean' and standardise the data. The next generation of clustering tools are likely to make extensive use of artificial intelligence in order to do this, in addition to also then drawing out insights between the clusters thus produced.

6. Dark Web monitoring

Dark Web content is the general name given to online material for which there are special access requirements; however in the context of online brand monitoring, it is usually taken to refer to content which is only accessible via the Tor network (a decentralised network involving the use of encrypted communications, and connections via multiple hops between Tor servers (proxies) - also known as relays or nodes). The Tor network - which is accessed using specially enabled browsers - can be used to view regular ('surface web') Internet content (and is one option open to users for whom anonymity is important), but is more usually used to access websites with the .onion extension, i.e. those which are only accessible from within the network^[13].

The Tor network of .onion websites includes a range of different content types, but is notorious for illegal and infringing content and, as such, can be a key area of interest for brand monitoring. However, many brand protection service providers offer only limited capabilities in this area. This is for a number of different reasons. One significant factor is that the Dark Web is essentially unregulated, frequently with no available links to 'real-world' contact details, and extremely limited enforcement options against infringing content. However, even in cases where takedown is not possible, intelligence on the content can be extremely valuable - one example may be on 'carder' websites, on which stolen financial credentials are traded; if (say) a financial services company can determine that the details for a particular credit card or bank account are being offered for sale, this provides the opportunity for the account to be 'locked' or deactivated.

It can also be extremely difficult to configure monitoring software to search the Dark Web. Whilst it is technically relatively straightforward to configure systems to be Tor-enabled (although connections are typically rather slow), there are generally no robust indexes of Dark Web content (such as the search engines and zone files used to search surface-web content), not least because the .onion addresses for any given website - which usually consist of long, random alphanumeric strings - are generally short-lived and change over time. A number of Dark Web search engines do exist, together with ad-hoc indexes of Dark Web content posted by users on sites such as Pastebin, but the information on these sources typically becomes out-of-date rather quickly.

The nature of the content on the Dark Web also means that security concerns can be an issue for brand-protection service providers wishing to build their capabilities in this area.

7. Mobile-based technologies

As Internet engagement has continued to grow over recent years, an increasing proportion of Internet use is conducted over mobile devices^[14,15], using a wide ecosystem of mobile apps. Many platforms are now almost exclusively mobile-based, often with little or no corresponding web presence - popular examples might include the WeChat / Weixin platforms, public groups on messaging services such as WhatsApp, and e-commerce platforms such as Pinduoduo. Many brand-protection service providers use legacy monitoring technologies which were designed specifically for analysing HTML content on the regular Internet and are often poorly equipped to address mobile technologies. In some cases, the work-around is to make use of standalone mobile devices or emulators - on which significant proportions of the monitoring is conducted manually - and there typically remains significant work to be done in order to fully integrate the relevant technologies into core monitoring capabilities.

8. Addressing the Web3 landscape

Web3 (also known as 'Web 3.0') is a general term referring to decentralised content on the Internet, with a particular focus on blockchain technologies. Blockchains are publicly accessible digital ledgers in which transactions are recorded, and form the basis of many digital currencies (or 'cryptocurrencies') (such as Bitcoin), in addition to a number of other applications, such as supply-chain control by brand owners. From a brand-protection viewpoint, the main related areas of interest are typically NFTs and blockchain domains^[16].

NFTs (non-fungible tokens) are digital files whose ownership is recorded on a blockchain. They are most commonly associated with graphics files (such as artworks and branded imagery) or other types of digital content (such as audio or music files). However, brand owners are increasingly incorporating NFTs into their business models, including areas such as the production and trade of virtual branded items (e.g. items to be worn by avatars in virtual-reality environments within the 'metaverse', the name given to a generalised connected environment of 3D virtual worlds). Consequently, unofficial branded NFTs can be a source of concern for brand owners.

Blockchain domains - which are recorded (together with their ownership details) on a blockchain, rather than using traditional registrars and web hosting - have a number of similarities to 'classic' domain names, and can be utilised in a number of ways. The most common uses are the creation of decentralised websites on peer-to-peer (P2P) platforms, to be accessed via specially-enabled browsers, or as addresses for sending and receiving cryptocurrency. However, the blockchain domain ecosystem is essentially unregulated, and nothing analogous to domain-name zone files is available. The system is made additionally more complicated by the fact the infrastructure allows for the possibility of domain-name 'clashes' - i.e. the potential for the same name to exist independently on distinct blockchains. As with traditional domain names, blockchain domains with brand-specific names can be threat to brand owners, and a potential source of confusion for customers.

Both NFTs and blockchain domains can be traded on NFT marketplaces (such as OpenSea), and the monitoring of these sites is typically the primary source of intelligence utilised by those brand-protection service providers offering capabilities in this area. For blockchain domains particularly, this approach is less than satisfactory, and offers nothing approaching the sort of comprehensive coverage as is available for regular gTLD domain names via zone-file analysis. Some additional information on the existence of registered blockchain domains is typically available through direct searches within databases provided by blockchain domain registrars and nameserver providers; however, the problem of more comprehensive detection is much more difficult to solve, potentially involving analysis of the content of the individual blockchains directly.

Another difficulty to be overcome in service offerings relating to NFTs and blockchain domains is the issue of enforcement against infringing content. In some cases, enforcement can be carried out through the submission of a DMCA (Digital Millennium Copyright Act) notice, and some NFT marketplaces have specific takedown procedures for content which infringes protected IP. However, in many cases, this simply involves the item being 'delisted' from the marketplace in question. In the future, we may see a move towards more rigorous enforcement, potentially involving forced transfers of ownership. Part of the problem is that the legal issues surrounding NFTs and blockchain domains are, in many cases, still not well-defined and are rapidly evolving, complicated by factors such as the fact that ownership of an NFT ownership does not necessarily grant ownership of copyright for the embedded content.

Beyond #8: Other emerging technologies

As new Internet technologies continue to emerge and develop, they will bring with them new risks for brand owners and associated challenges for brand-protection service providers, who will need to continue to observe and innovate in order to stay ahead of the curve.

At any given time, it is unclear where the next area of concern will come from. Currently, there is a great deal of buzz and speculation about artificial intelligence (AI) technologies and chatbots such as ChatGPT, but it is less obvious how these may affect brand-protection considerations. In this context, I am referring to content associated with, or produced by, AI applications. (Conversely, however, it seems highly likely that AI capabilities will be increasingly built into technologies used to facilitate the brand-protection process - i.e. tools to assist with monitoring, prioritisation, clustering and enforcement.)

Users are able to communicate with AI technologies such as ChatGPT via natural language, which are then able to construct responses based on information with which they have been 'trained'. This means that the information available from a chatbot is only as good as the data with which it has been trained (essentially, in the case of ChatGPT, including large volumes of Internet databases^[17,18]), and should really be treated with at least as much caution as the old "I'm Feeling Lucky" button on Google, where the user is just presented with a single response (not necessarily the most reliable one!) to any given query. This point is all the more valid given the ability of chatbots to extrapolate, and provide responses based on incomplete information. What this all means is that chatbots pose the risk of providing information about (say) a company or brand which is misleading or otherwise damaging to corporate reputation. However, since responses are generated dynamically in response to queries (rather than being 'fixed', as in the content of an HTML webpage), it is not clear how these issues might be addressed from a brand-protection point of view. Further complications surround issues such as the ownership of rights to content produced by AI technologies^[19].

Where chatbots may be of particular concern from a brand-protection and cybersecurity point of view is in their ability to rapidly create content of a wide variety of types, in a range of different styles - including the ability to write and de-bug computer code. What this may mean is that the entry barrier for infringers wishing to create compelling phishing e-mails^[20], or write malicious programs ('malware')^[21] may be significantly diminished. The likelihood is - at least in the first generations of AI technologies - that AI will not so much change the types of attack which are possible, but rather the ease with which they can be executed^[22].

Another issue surrounds use-cases in which AI systems are 'trained' with confidential corporate information as part of the process of creation of company materials (such as marketing releases). These scenarios raise the possibility for the information to be accessed by third parties, either directly via hacking, or via content included in the responses provided to other users, depending on the ways in which information is 'shared' within the infrastructure of the AI technology itself^[23]. 

References

[1] https://www.cst.cam.ac.uk/ring/halloffame

[2] https://www.markmonitor.com/download/ds/MarkMonitor-Corporate-Overview.pdf

[3] https://www.claymath.org/millennium-problems

[4] https://www.linkedin.com/pulse/assessing-mediating-digital-risk-landscape-brand-david-barnett/

[5] https://www.worldtrademarkreview.com/global-guide/anti-counterfeiting-and-online-brand-enforcement/2022/article/creating-cost-effective-domain-name-watching-programme

[6] https://www.cscdbs.com/blog/branded-domains-are-the-focal-point-of-many-phishing-attacks/

[7] https://www.cscdbs.com/en/resources-news/threatening-domains-targeting-top-brands/

[8] https://www.linkedin.com/pulse/hyphenated-domain-infringements-david-barnett/

[9] https://newgtlds.icann.org/en/about/program

[10] https://www.cscdbs.com/blog/the-world-of-the-subdomain/

[11] https://www.linkedin.com/pulse/exploring-domain-hostname-based-infringements-david-barnett/

[12] https://www.cscdbs.com/blog/do-you-see-what-i-see-geotargeting-in-brand-infringements/

[13] 'Brand Protection in the Online World: A Comprehensive Guide' by David Barnett (2016). Chapter 11: ''Deep' and 'Dark' Web'

[14] https://www.statista.com/statistics/617136/digital-population-worldwide/

[15] https://www.linkedin.com/pulse/holistic-brand-fraud-cyber-protection-using-domain-threat-barnett/

[16] https://www.linkedin.com/pulse/rise-nft-david-barnett

[17] https://www.sciencefocus.com/future-technology/gpt-3/

[18] https://techcrunch.com/2023/03/23/openai-connects-chatgpt-to-the-internet/

[19] https://intellectual-property-helpdesk.ec.europa.eu/news-events/news/intellectual-property-chatgpt-2023-02-20_en

[20] https://securityboulevard.com/2023/01/what-does-chat-gpt-imply-for-brand-impersonation-qa-with-dr-salvatore-stolfo/

[21] https://www.digitaltrends.com/computing/chatgpt-created-malware/

[22] https://venturebeat.com/security/security-risks-evolve-with-release-of-gpt-4/

[23] https://blogs.blackberry.com/en/2023/04/is-chatgpt-safe-for-organizations-to-use

This article was first published on 25 May 2023 at:

https://circleid.com/posts/20230525-the-millennium-problems-in-brand-protection

Entropy analysis of registered domain names relating to the top ten most valuable brands

Introduction and methodology

In our previous analysis^[1] we considered the use of the mathematical concept of Shannon entropy^[2] - essentially a measure of the amount of information (or 'randomness') in a domain name - as a way of clustering together related registrations, as part of the analysis process for identifying and prioritising threatening domains for enforcement and future monitoring. 

In this follow up, I extend the analysis, to consider domain registrations with names containing any of the top ten most valuable brands in 2022 (according to Interbrand)^[3]. The analysis considers all domains registered in a one-month period (from 28-Dec-2022 to 27-Jan-2023) and focuses only on those domains with names containing an exact match to the brand string, rather than considering typos and other fuzzy-match types. When considering trends and patterns in the dataset, I consider only active domains^[4] (i.e. where the most recent activity event is a registration or re-registration). This yields a dataset of 7,714 domains. For simplicity, I again also exclude any domains containing non-Latin characters (11 domains, or 0.14% of the total). The total numbers of results for each of the ten brands are shown in Table 1.

Brand string	Number of domain activity events[5] in monitoring period	Number of unique domains represented in dataset	Number of active domains as of date of analysis
apple	5,821	5,285	2,491
microsoft	707	639	314
amazon	4,768	4,186	1,847
google	1,924	1,844	777
samsung	608	566	244
toyota	1,140	1,114	566
coca(-)cola	65	63	29
mercedes	617	596	359
disney	1,032	971	456
nike	1,595	1,506	631

Table 1: Numbers of domain activity events and unique domains identified during the one-month monitoring period, and the numbers of active domains as of the date of analysis

Findings

Overall, the domains occupy a range of entropy values, from 1.722 (for a second-level domain name (SLD)^[6] of gogooglego) to 4.548 (for google-site-verificationuj64c5y9-rkbcpqaxydykjdrj1gop8tzij7nfxu). The set of top 60 names (i.e. those with entropy values of 3.892 or greater) encompasses all domain names within the dataset which appear visually to encompass long, apparently-random character strings (i.e. those which might typically arise in automated bulk registrations), and are listed in the Appendix. 

Seven of the top eight comprise very similar domain names (all targeting the Google brand, and with names beginning ‘google-site-verification’), on the .com and .net TLDs (top-level domains, or domain extensions), and are highly likely to represent a coordinated registration campaign, or to have been generated using similar automated registration algorithms. Note that, although all have similar (high) entropy values, the values are not identical (even amongst those with SLDs of the same length), because of the differing numbers of repeated characters in the apparently-random character strings. Although all seven domains are registered using a privacy-protection service, six of the seven appear explicitly to relate to the same entity (with the same 'Contact Privacy Inc.' customer number). These same contact details are actually given for 31 of the top 60 domains in the list, comprising what appears to be a significant cluster of related registrations, all registered via the same registrar (Google LLC), and targeting the Google (6 domains) and Microsoft (25 domains) brands^[7]. 

Other clusters of potentially related domain names are also apparent within the dataset, such as seven domains all with A-records associated with the same IP address, targeting the Amazon (4 domains), Microsoft (2 domains) and Disney (1 domain) brands, and all resolving to webpages monetised through the inclusion of pay-per-click (PPC) links.

The following other observations (all correct as of 27-Jan-2023) from the dataset of top 60 highest-entropy domain names are also of note:

• Only five of the 60 domains resolve to any significant site content. All of these relate to the Amazon brand, and have names featuring multiple keywords (with SLDs such as amazonproductreviewblog and amazonmysteryboxtruckload), rather than strings of apparently-random characters (suggesting deliberate choice of domain name, rather than automated registrations).

• 33 of the domains (55% of the total) display no active website, with the remainder resolving to generally low-threat content, such as pay-per-click (PPC) sites (11 domains; 18%), domain-for-sale pages (2 domains; 3%), or other placeholder pages or pages with no significant content. 

• 39 of the domains (65%) are configured with MX records, indicating that they are able to send and receive e-mails, and could be associated with phishing activity, even where no site content is present.

• 58 of the domains (97%) display no whois information, have redacted whois records, or use privacy-protection services. Although this is relatively common since the introduction of GDPR legislation, it can indicate an attempt by the domain owner to conceal their identity, and may be indicative of malicious use^[8,9].

• The dataset is dominated by domains registered via consumer-grade registrars (with a top five of: Google LLC (31 domains); GoDaddy.com, LLC (11); Tucows, Inc. (3); Wix.com Ltd. (3); Register.com, Inc. (2)), a trend which has also previously been noted for domains registered for infringing use^[10]. 

• 17 of the domains incorporate long (14 characters or more) apparently random strings of characters (including examples with SLDs such as 2cxqjwitvhtyh0-amazon, googlecb9c4560579f01d3 and microsoftexchange45e6e37e89e2e08e2e. Amongst the remainder of the dataset, there are numerous other domains featuring shorter apparently-random character strings or keyword patterns which may also be indicative of automated and/or bulk registrations (e.g. cloudworkflow-blv14-exec-microsoft365 and cloudworkflow-blv15-exec365-microsoft, or awsnetsuites-mailcloudroam01microsoftechowa and oauthnetsuites-mailcloudroam01promicrosoftechowa). 

Conclusion

The findings presented in this study highlight how trusted brands continue to be targeted by infringers, registering brand-specific domain names which may be intended for a range of malicious purposes. The analysis is also suggestive of the fact that these third parties appear to be utilising automated algorithms, to register large numbers of variant domain names - which may incorporate apparently random character strings - in bulk. This behaviour is consistent with the use of multiple short-lived domain names, to create hard-to-detect attacks such as those used in phishing, botnet creation, or other infringements, as has been noted in previous studies of practices such as domain tasting^[11,12]. These observations highlight the importance of ongoing, proactive brand- and domain monitoring and enforcement by brand owners.

The analysis also provides a further illustration of how the concept of domain-name entropy can be used as one criterion to cluster together related domains, on the basis of common features in their domain-name structure. For example, domain names registered using automated algorithms which generate long, random or pseudo-random character strings will tend to share similar or identical high entropy values. 

Appendix: Top 60 domain names in the dataset, by entropy values

SLD	TLD	Brand	SLD len. (chars)	Shannon entropy
google-site-verificationuj64c5y9-rkbcpqaxydykjdrj1gop8tzij7nfxu	net	google	63	4.548
google-site-verification2mxn0odchxhbhbphzzzkmv2rf3cyblumd6wvfbg	com	google	63	4.529
google-site-verificationdcgyugw3srf3zzz1anas0thyuegawdj2kcxniew	com	google	63	4.389
google-site-verificationngi9szdebvea6fop2-zalcux1sgrb7ozrdz4ump	com	google	63	4.342
notallowedscript63cef0c674953googlesyndication	com	google	46	4.320
google-site-verificationtszjjwyzw7wbjaotv59rpthglhhg4snojfrc4	com	google	61	4.310
google-site-verification0mqfjnhkxaxvtftff6psxtgzoqyatc-dszzzjq4	net	google	63	4.301
google-site-verificationiz8rx-lrwrfx8qmjryrlebc5e1acsia2ao8rhyd	net	google	63	4.279
amazon-ec3kh4ayn7a5a8czcza64jhstim448dd6namway746eca9549cjw0eea	com	amazon	63	4.210
logworkflow83microsoftexchange365	com	microsoft	33	4.208
workflow83-microsoftexchange365	com	microsoft	31	4.149
mail-workflow83-microsoftexchange365	com	microsoft	36	4.105
servworkflow83microsoftexchangeonline	com	microsoft	37	4.098
amz-haifhnakojankertyrewiqhhfoovpidjjjmicrosoft	net	microsoft	47	4.093
officeleadershipus02microsoftexchange	com	microsoft	37	4.091
microsoftexchange45e6e37e89e2e08e2e	com	microsoft	35	4.086
admin-mailcloudcitysend01workflowmicrosoftech	com	microsoft	45	4.084
cloudworkflow-blv14-exec10-microsoft365	com	microsoft	39	4.081
metanamegoogle-site-verificationcontenteszeodw7p1vrb4pj	com	google	55	4.081
fgwlockappsecurekjfdnamazon	com	amazon	27	4.060
amazonproductreviewblog	com	amazon	23	4.056
servworkflow83microsofttrade365	com	microsoft	31	4.051
log-workflow83-microsoftexchange365	com	microsoft	35	4.047
mail-workflow-microsoftexchange365	com	microsoft	34	4.023
admin-mailregistr2roam01workflowmicrosoftech	com	microsoft	44	4.018
cloudworkflows-blv14-exec-microsoft365	com	microsoft	38	4.015
bestamazonproducts2023	com	amazon	22	4.005
amazonkdpselfpublishing	com	amazon	23	4.002
cloudworkflow-blv14-exec-microsoft365	com	microsoft	37	4.000
payroll365microsoftdynamics	cloud	microsoft	27	3.986
amazonsecurityauth2023	com	amazon	22	3.971
amazonpublishingservice	com	amazon	23	3.969
gonetsuites-mailcloudroam01microsoftechpro	com	microsoft	42	3.959
officeleadership02-microsoftexchange	com	microsoft	36	3.953
bestsellingproductonamazon	com	amazon	26	3.950
bestsellingproductonamazon	co	amazon	26	3.950
oauthnetsuites-mailcloudroam01promicrosoftechowa	com	microsoft	48	3.949
cloudworkflow-blv15-exec365-microsoft	com	microsoft	37	3.946
amazonmysteryboxtruckload	com	amazon	25	3.943
skynetsuites-mailcloudtect01microsoftechserve	com	microsoft	45	3.942
microsoftpurviewday	com	microsoft	19	3.932
blueamazonfirestick	com	amazon	19	3.932
admin-mailcloudroam01workflowmicrosoftech	com	microsoft	41	3.927
metaversedisneyworldsmagickingdom	com	disney	33	3.923
amazonkdpselfpublish	com	amazon	20	3.922
mailcloudotp-10workflowhostmicrosoftech	com	microsoft	39	3.921
rivianamazondeliverytrucks	com	amazon	26	3.921
awsnetsuites-mailcloudroam01microsoftechowa	com	microsoft	43	3.919
amazonpublishingstore	com	amazon	21	3.916
microsoft365emailplus	com	microsoft	21	3.916
amazonpublishingworld	com	amazon	21	3.916
amz-haifhnakoajdjbfhiswiqhhfoovpidjjjmicrosoft	net	microsoft	46	3.916
googlecb9c4560579f01d3	com	google	22	3.914
thewbialtdisneycompany	com	disney	22	3.914
applegatekitchensandbathrooms	uk	apple	29	3.909
applegatekitchensandbathrooms	co.uk	apple	29	3.909
partnetsuites-mailcloudroar01microsoftechpros	com	microsoft	45	3.898
2cxqjwitvhtyh0-amazon	com	amazon	21	3.897
fvnexwopsdqb21-amazon	com	amazon	21	3.897
mailweb23-microsofts365	com	microsoft	23	3.892

References

[1] https://www.linkedin.com/pulse/investigating-use-domain-name-entropy-clustering-results-barnett/

[2] https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf

[3] https://interbrand.com/best-global-brands-2022-download-form/; the brand strings considered are: 'apple', 'microsoft', 'amazon', 'google', 'samsung', 'toyota', 'coca(-)cola', 'mercedes', 'disney', 'nike'

[4] As of the date of analysis (27-Jan-2023)

[5] A domain activity event is defined as an instance of a registration, re-registration, or domain drop (lapse)

[6] The SLD is the part of the domain name before the dot

[7] Note that, in general, even if two domain names targeting two different brands utilised identical styles of apparently-random character strings, we would not necessarily expect the domains to have identical entropy values, because the lengths and structures of the brand strings themselves may differ

[8] https://www.cscdbs.com/en/resources-news/supply-chain-report/

[9] https://www.cscdbs.com/en/resources-news/threatening-domains-targeting-top-brands/

[10] https://www.cscdbs.com/en/resources-news/impact-of-covid-on-internet-security/

[11] https://www.cscdbs.com/blog/patterns-and-trends-in-domain-tasting-of-the-top-10-global-brands/

[12] https://www.linkedin.com/pulse/patterns-trends-domain-tasting-top-ten-global-brands-david-barnett/

This article was first published on 2 February 2023 at:

https://www.linkedin.com/pulse/entropy-analysis-registered-domain-names-relating-top-david-barnett/

Investigating the use of domain-name entropy for clustering results

by Lan Huang and David Barnett

Introduction

The importance of being able to 'cluster' together similar or connected brand infringements has been noted in numerous studies^[1]. Clustering has a number of benefits, including the ability to identify serial infringers for prioritised enforcement action, reveal instances of bad-faith activity, and providing the potential for efficient bulk enforcement actions. 

A key associated idea is the concept of quantifying threat - i.e. determining which domains (or other results) may pose the greatest potential for infringing use in the future, even where no content is currently present - allowing prioritisation of results for initial analysis, enforcement or tracking for content changes. 

Expanding on these ideas, previous work^[2] has revealed that large coordinated infringement or attack campaigns (such as the registration of domains for use in spamming activity, malware distribution or botnet creation) are often associated with batches of domains purchased through registrars who offer easy access to bulk registrations using automated algorithms. These registrations can be generated via automated recommendations by the registrar, or through the upload of lists of requested domain names. In many cases, the domain names used for these purposes may contain no meaningful keywords (appearing just as random strings of characters), and may be very long. It is also noteworthy that the use of (pseudo-)random domain names may be beneficial to bad actors, as they are unlikely to contain brand terms and are therefore more difficult to detect using classic brand-monitoring techniques.

In order to explore domain registrations of this type, we utilise the concept of Shannon entropy^[3]. This is a mathematical concept in information theory, used to quantify the amount of information (or 'surprise') stored in a string or, equivalently, the number of bits needed to optimally encode the string (i.e. a lower bound). In this study, we apply the idea to domain names by calculating the Shannon entropy associated with the second-level domain (SLD) name string^{[cf. 4]} (i.e. the part of the domain name before the dot, and excluding the TLD (domain extension))^[5]. Broadly, this means that domain names which are short and/or have large numbers of repeated characters will have low entropy, and domain names which are longer and/or contain large numbers of distinct characters will have high entropy. Our hypothesis is that a batch of domains registered for a coordinated campaign, with a specific algorithm used to generate the domain names, will tend to be clusterable together on the basis that they will share a common date of registration, common registrar, and will have similar entropy values. Overall, long random domain-name strings associated with automated registrations will tend to have high entropy values. 

Methodology and analysis

In order to look more closely at these ideas, we consider the case study of all domains registered on a particular day (13-Dec-2022), using zone-file information. This dataset consists of approximately 205,000 domains - however, for simplicity we exclude from the analysis those featuring non-Latin characters (i.e. Punycode, or homoglyph, domains - accounting for 0.6% of the total), and focus on the remainder, consisting of domain names containing the characters a-z, 0-9, and the hyphen ('-'). 

Across the dataset, the domains are associated with a range of entropy values, from 0.000 to 4.700, as shown in Figure 1.

Figure 1: Distribution of Shannon entropy values for the set of all (non-Punycode) domains registered on a single day (divided into entropy-value 'bins' of width 0.1)

The top and bottom domain names in the dataset (by entropy values) are shown in Tables 1 and 2.

SLD-name string	TLD (domain extension)	SLD length (chars.)	Shannon entropy
abcdefghijklmnopqrstuvwxyz	space	26	4.700
viqxacb7wo6l3hfujw3agf3stcce6eenl4kovfza3rzri4gwyxg6auid	com	56	4.642
b4su4qo65fkefg3cpd5muxwekbn4vx6fr7ieroavxqwco2xrqmrrwlad	com	56	4.591
oz5winfavnvbmgdspa633wdnpmbjjrp6crwutyt4uxgxkvytbjdmdc	com	54	4.569
hydraclubbioknikokex7njhwuahc2l67lfiz7z36md2jvopda7nchidshop	com	60	4.550
q374uuwdlgtkveh2acqi6ubhic4m3bnwb32kc2yqmxf2ilv36leujnid	com	56	4.539
mekck2mf2uju3ssjl2woyddfrunwcnevfql3imp4tfr3z6wmjmo4jvid	com	56	4.497
facebook-domain-verificationyx7q3wstorn4idf9xqtzdz842q0b6x	com	58	4.475
vh6bjre5lw9iuegs1b9fspitswrdnbtsm1emunvlulbo6uc0	top	48	4.470
skjcd-98729871cnf5bnb8ewr2e-vq438vnjy0mtg1mdcumty2n	xyz	51	4.464
a3n3mq7c3xl7u4mfvhhjyjz2x7lqd7sf5jfm66mhf33fxlyodb5pibyd	com	56	4.455
osli77ygq5myyquqzc2sva7wgnjc2m7yozz67k3kkgkrync4puw3cqyd	com	56	4.439
7tl2qxwot624do6kbkvqwsg6knaz6jnlx5kfktni7bzt3qlo4imk4tqd	com	56	4.412
q6g5o01vsfyw95all7x1krjdki	com	26	4.393
12mnbvcxzasdfghjklpoi	com	21	4.392
qwertyuiop12asdfghjkkl	com	22	4.369
owsyeuxoyy4qtm4bkazrkxjtzhydedxgoxkd2yqddmxgcjevmhnbenyd	com	56	4.327
metanamepdomainverifycontent38656d7bbe27c23f182336255	com	53	4.306
fqskypondteieqxoxgizamgqrwlb	info	28	4.280
zaqwsxcderfvbgtgbnhy12	com	22	4.278
vij8q5xcentralr2hm910v	sbs	22	4.278

Table 1: Top domain names by entropy values

SLD-name string	TLD (domain extension)	SLD length (chars.)	Shannon entropy
n	camp	1	0.000
d	supplies	1	0.000
s	camera	1	0.000
4	flights	1	0.000
n	reise	1	0.000
n	clinic	1	0.000
0	condos	1	0.000
9	events	1	0.000
9	photography	1	0.000
rr	center	2	0.000
cc	degree	2	0.000
999	guide	3	0.000
7777	best	4	0.000
44444	tel	5	0.000
88888	tel	5	0.000
ooooo	events	5	0.000
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv	vip	63	0.000
00-00	co.uk	5	0.258
000000004	xyz	9	0.503
000000002	xyz	9	0.503
000000002	com	9	0.503

Table 2: Bottom domain names by entropy values

In order to look more closely at the characteristics of the highest-entropy sites, we conducted a deeper dive into the top 1,000 domain names within the dataset (those with entropy values of 3.823 and above, incorporating the majority of the examples which appear visually to consist of apparently 'random' strings).

Of these top 1,000 domain names:

• 847 (84.7%) have active A records (i.e. are associated with a specific IP address, and potentially live website content)
• 275 (27.5%) have active MX records (i.e. are configured to be able to send and receive e-mails and which may therefore be associated with phishing activity)
• 777 (77.7%) use domain privacy services or have redacted registration information (see also Table 3) (demonstrating the domain owners' attempts to mask their identity and which could indicate nefarious intentions^[6]).

The top registrants and registrars represented within the dataset (for those domains where whois information is available) are shown in Tables 3 and 4.

Registrant	No. domains
Domains By Proxy, LLC	219
REDACTED FOR PRIVACY	197
Privacy service provided by Withheld for Privacy ehf	82
Contact Privacy Inc. Customer 7151571251	75
c/o whoisproxy.com	36
Privacy Protect, LLC (PrivacyProtect.org)	29
Wix.com Ltd.	15
PrivacyGuardian.org llc	11
Data Protected	9
Domain Protection Services, Inc.	7

Table 3: Top registrants

Registrant	No. domains
GoDaddy.com LLC	219
DYNADOT LLC	128
Google LLC	78
NAMECHEAP INC	78
TUCOWS, INC.	53
Key-Systems GmbH	43
Wix.Com Ltd.	30
PDR Ltd. d/b/a PublicDomainRegistry.com	16
Tucows Domains Inc.	13
Atak Domain Hosting	12
Ionos SE	12

Table 4: Top registrars

The top registrars within the dataset are almost exclusively consumer-grade registrars, as would be expected for domains associated with automated registrations for infringing use, and which mirrors the landscape seen in other studies of infringing and potentially threatening domains^[7,8]. Furthermore, just a small number of registrars accounts for the vast majority of the high-entropy domain names; the top eleven shown in Table 4 together account for 682 of the domains within the dataset (i.e. 68.2% of the total). 

Furthermore, the dataset does incorporate the types of domain clusters we might expect to see arising from automated registrations for infringing use. The best example is a set of 125 domains with the following characteristics:

• All with SLD names consisting of apparently-random 15-character alphanumeric strings, and with identical entropy values (all 3.907^[9])
• All hosted on the .buzz extension (one of the top thirty overall highest-threat TLDs, according to a recent CSC study^[10])
• All registered through Dynadot LLC with redacted whois records
• (Of the 114 with active A records), associated with just 5 distinct IP addresses, most of which are in similar netblocks

It is highly likely that this cluster comprises a coordinated registration event by a single entity, and may well have been registered with intention of use for threatening or infringing activity. 

As of the date of analysis (15-Dec-2022; two days after registration), the domains resolved to a mix of Chinese-language gambling-portal pages (see Figure 2) and dead pages. Whilst this may be the ultimate intended content (potentially as part of a revenue-generating affiliate scheme), it could also be just 'placeholder' content, uploaded until the sites are weaponised for higher-threat purposes, or may be material designed to be visible only from certain geographical regions to mask the 'real' content and evade detection (so-called 'geotargeted' content^[11]). In any case, these sites may certainly warrant further monitoring for changes.

Figure 2: Examples of webpage content visible within the cluster of related .buzz domain names

Conclusion

This analysis highlights how the determination of entropy values for the SLD-name strings of registered domain names can be a valuable component of algorithms to determine which examples are most likely to be intended for infringing or fraudulent use. The calculation can also help to link together clusters of related domain names, to build up a picture of activity by specific bad actors, even in cases where the individual whois records are redacted.

These ideas can be applied in the development of technology allowing brand owners to identify key threat vectors and areas of risk, and determine where mediating action is most urgently required.

References

[1] https://www.linkedin.com/pulse/holistic-brand-fraud-cyber-protection-using-domain-threat-barnett/

[2] https://interisle.net/sub/CriminalDomainAbuse.pdf

[3] https://arxiv.org/ftp/arxiv/papers/1405/1405.2061.pdf

[4] https://www.farsightsecurity.com/blog/txt-record/automatingdetection-20190517/ 

[5]  The Shannon entropy (H) of the SLD-name string is calculated as:

H = - Σ_i [ p_i × log₂(p_i) ] 

where p_i is the proportion of the string made of the i^th character (the 'probability'). The summation is carried out over the pool of possible characters. 

[6] https://www.cscdbs.com/en/resources-news/supply-chain-report/

[7] https://www.cscdbs.com/en/resources-news/impact-of-covid-on-internet-security/

[8] https://www.cscdbs.com/en/resources-news/threatening-domains-targeting-top-brands/

[9]  This value arises because, rather than actually being truly random, the SLD names in all cases consist of 15 distinct characters. Therefore: 

H = - 15 × [ (1/15) × log₂(1/15) ] = log₂(15) = 3.907

[10] https://www.cscdbs.com/blog/the-highest-threat-tlds-part-2/

[11] https://www.cscdbs.com/blog/do-you-see-what-i-see-geotargeting-in-brand-infringements/

This article was first published on 2 February 2023 at:

https://www.linkedin.com/pulse/investigating-use-domain-name-entropy-clustering-results-barnett/