Natural language processing (NLP) is the processing of natural language information by a computer. NLP is a subfield of computer science and is closely associated with artificial intelligence. NLP is also related to information retrieval, knowledge representation, computational linguistics, and linguistics more broadly. Major processing tasks in an NLP system include: speech recognition, text classification, natural language understanding, and natural language generation. == History == Natural language processing has its roots in the 1950s. Already in 1950, Alan Turing published an article titled "Computing Machinery and Intelligence," which proposed what is now called the Turing test as a criterion of intelligence, though at the time that was not articulated as a problem separate from artificial intelligence. The proposed test includes a task that involves the automated interpretation and generation of natural language. === Symbolic NLP (1950s – early 1990s) === The premise of symbolic NLP is often illustrated using John Searle's Chinese room thought experiment: Given a collection of rules (e.g., a Chinese phrasebook, with questions and matching answers), the computer emulates natural language understanding (or other NLP tasks) by applying those rules to the data it confronts. 1950s: The Georgetown experiment in 1954 involved fully automatic translation of more than sixty Russian sentences into English. The authors claimed that within three or five years, machine translation would be a solved problem. However, real progress was much slower, and after the ALPAC report in 1966, which found that ten years of research had failed to fulfill the expectations, funding for machine translation was dramatically reduced. Little further research in machine translation was conducted in America (though some research continued elsewhere, such as Japan and Europe) until the late 1980s when the first statistical machine translation systems were developed. 1960s: Some notably successful natural language processing systems developed in the 1960s were SHRDLU, a natural language system working in restricted "blocks worlds" with restricted vocabularies, and ELIZA, a simulation of Rogerian psychotherapy, written by Joseph Weizenbaum between 1964 and 1966. Despite using minimal information about human thought or emotion, ELIZA was able to produce interactions that appeared human-like. When the "patient" exceeded the very small knowledge base, ELIZA might provide a generic response, for example, responding to "My head hurts" with "Why do you say your head hurts?". Ross Quillian's successful work on natural language was demonstrated with a vocabulary of only twenty words, because that was all that would fit in a computer memory at the time. 1970s: During the 1970s, many programmers began to write "conceptual ontologies", which structured real-world information into computer-understandable data. Examples are MARGIE (Schank, 1975), SAM (Cullingford, 1978), PAM (Wilensky, 1978), TaleSpin (Meehan, 1976), QUALM (Lehnert, 1977), Politics (Carbonell, 1979), and Plot Units (Lehnert 1981). During this time, the first chatterbots were written (e.g., PARRY). 1980s: The 1980s and early 1990s mark the heyday of symbolic methods in NLP. Focus areas of the time included research on rule-based parsing (e.g., the development of HPSG as a computational operationalization of generative grammar), morphology (e.g., two-level morphology), semantics (e.g., Lesk algorithm), reference (e.g., within Centering Theory) and other areas of natural language understanding (e.g., in the Rhetorical Structure Theory). Other lines of research were continued, e.g., the development of chatterbots with Racter and Jabberwacky. An important development (that eventually led to the statistical turn in the 1990s) was the rising importance of quantitative evaluation in this period. === Statistical NLP (1990s–present) === Up until the 1980s, most natural language processing systems were based on complex sets of hand-written rules. Starting in the late 1980s, however, there was a revolution in natural language processing with the introduction of machine learning algorithms for language processing. This shift was influenced by increasing computational power (see Moore's law) and a decline in the dominance of Chomskyan linguistic theories (e.g. transformational grammar), whose theoretical underpinnings discouraged the sort of corpus linguistics that underlies the machine-learning approach to language processing. 1990s: Many of the notable early successes in statistical methods in NLP occurred in the field of machine translation, due especially to work at IBM Research, such as IBM alignment models. These systems were able to take advantage of existing multilingual textual corpora that had been produced by the Parliament of Canada and the European Union as a result of laws calling for the translation of all governmental proceedings into all official languages of the corresponding systems of government. However, many systems relied on corpora that were specifically developed for the tasks they were designed to perform. This reliance has been a major limitation to their broader effectiveness and continues to affect similar systems. Consequently, significant research has focused on methods for learning effectively from limited amounts of data. 2000s: With the growth of the web, increasing amounts of raw (unannotated) language data have become available since the mid-1990s. Research has thus increasingly focused on unsupervised and semi-supervised learning algorithms. Such algorithms can learn from data that has not been hand-annotated with the desired answers or using a combination of annotated and non-annotated data. Generally, this task is much more difficult than supervised learning, and typically produces less accurate results for a given amount of input data. However, large quantities of non-annotated data are available (including, among other things, the entire content of the World Wide Web), which can often make up for the worse efficiency if the algorithm used has a low enough time complexity to be practical. 2003: word n-gram model, at the time the best statistical algorithm, is outperformed by a multi-layer perceptron (with a single hidden layer and context length of several words, trained on up to 14 million words, by Bengio et al.) 2010: Tomáš Mikolov (then a PhD student at Brno University of Technology) with co-authors applied a simple recurrent neural network with a single hidden layer to language modeling, and in the following years he went on to develop Word2vec. In the 2010s, representation learning and deep neural network-style (featuring many hidden layers) machine learning methods became widespread in natural language processing. This shift gained momentum due to results showing that such techniques can achieve state-of-the-art results in many natural language tasks, e.g., in language modeling and parsing. This is increasingly important in medicine and healthcare, where NLP helps analyze notes and text in electronic health records that would otherwise be inaccessible for study when seeking to improve care or protect patient privacy. == Approaches: Symbolic, statistical, neural networks == Symbolic approach, i.e., the hand-coding of a set of rules for manipulating symbols, coupled with a dictionary lookup, was historically the first approach used both by AI in general and by NLP in particular: such as by writing grammars or devising heuristic rules for stemming. Machine learning approaches, which include both statistical and neural networks, on the other hand, have many advantages over the symbolic approach: both statistical and neural network methods tend to focus more on the most common cases extracted from a corpus of texts, whereas the rule-based approach needs to provide rules for both rare and common cases equally. language models, produced by either statistical or neural networks methods, are more robust to both unfamiliar (e.g. containing words or structures that have not been seen before) and erroneous input (e.g. with misspelled words or words accidentally omitted) in comparison to the rule-based systems, which are also more costly to produce. the larger such a (probabilistic) language model is, the more accurate it becomes, in contrast to rule-based systems that can gain accuracy only by increasing the amount and complexity of the rules leading to intractability problems. Rule-based systems are commonly used: when the amount of training data is insufficient to successfully apply machine learning methods, e.g., for the machine translation of low-resource languages such as provided by the Apertium system, for preprocessing in NLP pipelines, e.g., tokenization, or for post-processing and transforming the output of NLP pipelines, e.g., for knowledge extraction from syntactic parses. === Statistical approach === In the late 1980s and mid-1990s, the statistical approach ended a peri
Lost Art-Database
The Lost Art-Datenbank is an online database published by the German Lost Art Foundation (Deutsches Zentrum Kulturgutverluste. It contains information on cultural objects looted from Jewish collectors or transferred due to Nazi persecution during the Nazi era. Until 2015, it was managed by the Koordinierungsstelle für Kulturgutverluste (Magdeburg Coordination Office). == Creation == Following the Washington Conference of 1998, and the commitments to provide more transparency regarding looted art, Germany launched the Lost Art Database in 2000 order to help Holocaust victims and their families track down artworks that had been looted from them or lost due to Nazi persecution. == Functionality == The Lost Art Database lists art and books and other cultural objects that were lost, seized, stolen or forceably sold during the Nazi era. The database is divided into search requests from victims' families, heirs or institutions and "found" reports from cultural institutions on items with unresolved provenance gaps from the Nazi periods. The section on reports of finds lists objects that are known to have been unlawfully seized or relocated as a result of the war. In addition, reports are published here on cultural objects for which an uncertain or incomplete provenance may indicate a possible unlawful seizure or war-related relocation. The publication of reports in the Lost Art Internet Database is carried out on behalf of and with the consent of the reporting persons and institutions. The responsibility for the content of the reports lies with these legal or natural persons. There have been controversies over which items should be included in the database. Lost Art is based on the Washington Principles adopted in 1998, which Germany has committed itself to implementing (Joint Declaration, 1999). The Lost Art Database is considered a key resource in the search for looted art and the victims of persecution. Every item in the Lost Art Database has an identifier, known as a Lost Art ID. Proveana is the linked research database. == Other lost art databases == Other countries have launched databases to help identify Nazi looted art. Each database has its own area of focus. The German Lost Art Database allows families or heirs to submit information. Other countries have databases that focus on looted artworks that have not been found or artworks that were repatriated to the national authorities after the defeat of the Nazis but were never returned to their original owners. Other databases have been created for stolen antiquities, looted art from colonial era, art stolen from Syria, Iraq, Ukraine, or from museums or collectors.
Pepper (cryptography)
In cryptography, a pepper is a secret added to an input such as a password during hashing with a cryptographic hash function. This value differs from a salt in that it is not stored alongside a password hash, but rather the pepper is kept separate using another meachanism, such as a Hardware Security Module. Note that the National Institute of Standards and Technology refers to this value as a secret key rather than a pepper. A pepper is similar in concept to a salt or an encryption key. It is like a salt in that it is a randomized value that is added to a password hash, and it is similar to an encryption key in that it should be kept secret. A pepper performs a comparable role to a salt or an encryption key, but while a salt is not secret (merely unique) and can be stored alongside the hashed output, a pepper is secret and must not be stored with the output. The hash and salt are usually stored in a database, but, if stored, a pepper must be stored separately to prevent it from being obtained by the attacker in case of a database breach. == History == The idea of a site- or service-specific salt (in addition to a per-user salt) has a long history, with Steven M. Bellovin proposing a local parameter in a Bugtraq post in 1995. In 1996 Udi Manber also described the advantages of such a scheme, terming it a secret salt. However, he suggested not storing the value of the secret salt, but instead rediscovering it by trial and error at password verification time. The term pepper has been used, by analogy to salt, but with a variety of meanings. For example, when discussing a challenge-response scheme, pepper has been used for a salt-like quantity, though not used for password storage; it has been used for a data transmission technique where a pepper must be guessed; and even as a part of jokes. The term pepper was proposed for a secret or local parameter stored separately from the password in a discussion of protecting passwords from rainbow table attacks. This usage did not immediately catch on: for example, Fred Wenzel added support to Django password hashing for storage based on a combination of bcrypt and HMAC with separately stored nonces, without using the term. Usage has since become more common. == Types == There are multiple different types of pepper: A shared secret that is common to all users. A randomly-selected number that must be re-discovered on every password input. These mechanisms could be combined with password salting, iterated hashing or even one another. == Shared-secret pepper == Bellovin and Webster suggest prepend a shared secret to the password before hashing, which allows easy use of existing hash functions. For example, consider two users to be added to a database. This table contains two combinations of username and password. The password is not saved, and the 8-byte (64-bit) 44534C70C6883DE2 pepper is saved in a safe place separate from the output values of the hash, in this case SHA256. Unlike the salt, the pepper does not provide protection to users who use the same password, but protects against dictionary attacks, unless the attacker has the pepper value available. Since the same pepper is not shared between different applications, an attacker is unable to reuse the hashes of one compromised database to another. A complete scheme for saving passwords may include both salt and pepper use. For example, it has been suggested to combine the pepper by encrypting salted password hashes, which allows rotation of the pepper. In the case of a shared-secret pepper, a single compromised password (via password reuse or other attack) along with a user's salt can lead to an attack to discover the pepper, rendering it ineffective. If an attacker knows a plaintext password and a user's salt, as well as the algorithm used to hash the password, then discovering the pepper can be a matter of brute forcing the values of the pepper. This is why NIST recommends the secret value be at least 112 bits, so that discovering it by exhaustive search is prohibitively expensive. The pepper must be generated anew for every application it is deployed in, otherwise a breach of one application would result in lowered security of another application. Without knowledge of the pepper, other passwords in the database will be far more difficult to extract from their hashed values, as the attacker would need to guess the password as well as the pepper. A pepper adds security to a database of salts and hashes because unless the attacker is able to obtain the pepper, cracking even a single hash is intractable, no matter how weak the original password. Even with a list of (salt, hash) pairs, an attacker must also guess the secret pepper in order to find the password which produces the hash. The NIST specification for a secret salt suggests using a Password-Based Key Derivation Function (PBKDF) with an approved Pseudorandom Function such as HMAC with SHA-3 as the hash function of the HMAC. The NIST recommendation is also to perform at least 1000 iterations of the PBKDF, and a further minimum 1000 iterations using the secret salt in place of the non-secret salt. == Randomly-selected pepper that must be re-discovered == The aim of this mechanism is to slow down password the password verification step, thus slowing attacks. The aim is similar increasing the iteration count on bcrypt or Argon2, but the mechanism is different. The secret salt or pepper must be rediscovered by the verifier or attacker each time by guessing. In this situation, the password hashing function is calculated using both the password and the pepper. At password storage time, the pepper is chosen randomly from a range between 1 and R, the hash output is calculated using the password and the pepper. The hash output is stored with the username. The pepper is then discarded. At password verification time, the verifier is provided with a username and password to verify. The originally calculated hash is retrieved for the given username, and then the hash of the password and each value between 1 and R is calculated. If any of these hash values match the stored password hash, the password is considered valid. Note, the possible values of the pepper should not be tested in a fixed order known to an attacker, otherwise a timing attack may reveal the pepper. If the password is correct, the correct pepper will be found in R/2 hash evaluations on average. If the password is incorrect, all R values must be tested before the password can be rejected.
Contrast set learning
Contrast set learning is a form of association rule learning that seeks to identify meaningful differences between separate groups by reverse-engineering the key predictors that identify for each particular group. For example, given a set of attributes for a pool of students (labeled by degree type), a contrast set learner would identify the contrasting features between students seeking bachelor's degrees and those working toward PhD degrees. == Overview == A common practice in data mining is to classify, to look at the attributes of an object or situation and make a guess at what category the observed item belongs to. As new evidence is examined (typically by feeding a training set to a learning algorithm), these guesses are refined and improved. Contrast set learning works in the opposite direction. While classifiers read a collection of data and collect information that is used to place new data into a series of discrete categories, contrast set learning takes the category that an item belongs to and attempts to reverse engineer the statistical evidence that identifies an item as a member of a class. That is, contrast set learners seek rules associating attribute values with changes to the class distribution. They seek to identify the key predictors that contrast one classification from another. For example, an aerospace engineer might record data on test launches of a new rocket. Measurements would be taken at regular intervals throughout the launch, noting factors such as the trajectory of the rocket, operating temperatures, external pressures, and so on. If the rocket launch fails after a number of successful tests, the engineer could use contrast set learning to distinguish between the successful and failed tests. A contrast set learner will produce a set of association rules that, when applied, will indicate the key predictors of each failed tests versus the successful ones (the temperature was too high, the wind pressure was too high, etc.). Contrast set learning is a form of association rule learning. Association rule learners typically offer rules linking attributes commonly occurring together in a training set (for instance, people who are enrolled in four-year programs and take a full course load tend to also live near campus). Instead of finding rules that describe the current situation, contrast set learners seek rules that differ meaningfully in their distribution across groups (and thus, can be used as predictors for those groups). For example, a contrast set learner could ask, “What are the key identifiers of a person with a bachelor's degree or a person with a PhD, and how do people with PhD's and bachelor’s degrees differ?” Standard classifier algorithms, such as C4.5, have no concept of class importance (that is, they do not know if a class is "good" or "bad"). Such learners cannot bias or filter their predictions towards certain desired classes. As the goal of contrast set learning is to discover meaningful differences between groups, it is useful to be able to target the learned rules towards certain classifications. Several contrast set learners, such as MINWAL or the family of TAR algorithms, assign weights to each class in order to focus the learned theories toward outcomes that are of interest to a particular audience. Thus, contrast set learning can be thought of as a form of weighted class learning. === Example: Supermarket Purchases === The differences between standard classification, association rule learning, and contrast set learning can be illustrated with a simple supermarket metaphor. In the following small dataset, each row is a supermarket transaction and each "1" indicates that the item was purchased (a "0" indicates that the item was not purchased): Given this data, Association rule learning may discover that customers that buy onions and potatoes together are likely to also purchase hamburger meat. Classification may discover that customers that bought onions, potatoes, and hamburger meats were purchasing items for a cookout. Contrast set learning may discover that the major difference between customers shopping for a cookout and those shopping for an anniversary dinner are that customers acquiring items for a cookout purchase onions, potatoes, and hamburger meat (and do not purchase foie gras or champagne). == Treatment learning == Treatment learning is a form of weighted contrast-set learning that takes a single desirable group and contrasts it against the remaining undesirable groups (the level of desirability is represented by weighted classes). The resulting "treatment" suggests a set of rules that, when applied, will lead to the desired outcome. Treatment learning differs from standard contrast set learning through the following constraints: Rather than seeking the differences between all groups, treatment learning specifies a particular group to focus on, applies a weight to this desired grouping, and lumps the remaining groups into one "undesired" category. Treatment learning has a stated focus on minimal theories. In practice, treatment are limited to a maximum of four constraints (i.e., rather than stating all of the reasons that a rocket differs from a skateboard, a treatment learner will state one to four major differences that predict for rockets at a high level of statistical significance). This focus on simplicity is an important goal for treatment learners. Treatment learning seeks the smallest change that has the greatest impact on the class distribution. Conceptually, treatment learners explore all possible subsets of the range of values for all attributes. Such a search is often infeasible in practice, so treatment learning often focuses instead on quickly pruning and ignoring attribute ranges that, when applied, lead to a class distribution where the desired class is in the minority. === Example: Boston housing data === The following example demonstrates the output of the treatment learner TAR3 on a dataset of housing data from the city of Boston (a nontrivial public dataset with over 500 examples). In this dataset, a number of factors are collected for each house, and each house is classified according to its quality (low, medium-low, medium-high, and high). The desired class is set to "high", and all other classes are lumped together as undesirable. The output of the treatment learner is as follows: Baseline class distribution: low: 29% medlow: 29% medhigh: 21% high: 21% Suggested Treatment: [PTRATIO=[12.6..16), RM=[6.7..9.78)] New class distribution: low: 0% medlow: 0% medhigh: 3% high: 97% With no applied treatments (rules), the desired class represents only 21% of the class distribution. However, if one filters the data set for houses with 6.7 to 9.78 rooms and a neighborhood parent-teacher ratio of 12.6 to 16, then 97% of the remaining examples fall into the desired class (high-quality houses). == Algorithms == There are a number of algorithms that perform contrast set learning. The following subsections describe two examples. === STUCCO === The STUCCO contrast set learner treats the task of learning from contrast sets as a tree search problem where the root node of the tree is an empty contrast set. Children are added by specializing the set with additional items picked through a canonical ordering of attributes (to avoid visiting the same nodes twice). Children are formed by appending terms that follow all existing terms in a given ordering. The formed tree is searched in a breadth-first manner. Given the nodes at each level, the dataset is scanned and the support is counted for each group. Each node is then examined to determine if it is significant and large, if it should be pruned, and if new children should be generated. After all significant contrast sets are located, a post-processor selects a subset to show to the user - the low order, simpler results are shown first, followed by the higher order results which are "surprising and significantly different." The support calculation comes from testing a null hypothesis that the contrast set support is equal across all groups (i.e., that contrast set support is independent of group membership). The support count for each group is a frequency value that can be analyzed in a contingency table where each row represents the truth value of the contrast set and each column variable indicates the group membership frequency. If there is a difference in proportions between the contrast set frequencies and those of the null hypothesis, the algorithm must then determine if the differences in proportions represent a relation between variables or if it can be attributed to random causes. This can be determined through a chi-square test comparing the observed frequency count to the expected count. Nodes are pruned from the tree when all specializations of the node can never lead to a significant and large contrast set. The decision to prune is based on: The minimum deviation size: The maximum difference between the support
Data Reference Model
The Data Reference Model (DRM) is one of the five reference models of the Federal Enterprise Architecture. == Overview == The DRM is a framework whose primary purpose is to enable information sharing and reuse across the United States federal government via the standard description and discovery of common data and the promotion of uniform data management practices. The DRM describes artifacts which can be generated from the data architectures of federal government agencies. The DRM provides a flexible and standards-based approach to accomplish its purpose. The scope of the DRM is broad, as it may be applied within a single agency, within a community of interest, or cross-community of interest. == Data Reference Model topics == === DRM structure === The DRM provides a standard means by which data may be described, categorized, and shared. These are reflected within each of the DRM's three standardization areas: Data Description: Provides a means to uniformly describe data, thereby supporting its discovery and sharing. Data Context: Facilitates discovery of data through an approach to the categorization of data according to taxonomies. Additionally, enables the definition of authoritative data assets within a community of interest. Data Sharing: Supports the access and exchange of data where access consists of ad hoc requests (such as a query of a data asset), and exchange consists of fixed, re-occurring transactions between parties. Enabled by capabilities provided by both the Data Context and Data Description standardization areas. === DRM Version 2 === The Data Reference Model version 2 released in November 2005 is a 114-page document with detailed architectural diagrams and an extensive glossary of terms. The DRM also make many references to ISO standards specifically the ISO/IEC 11179 metadata registry standard. === DRM usage === The DRM is not technically a published technical interoperability standard such as web services, it is an excellent starting point for data architects within federal and state agencies. Any federal or state agencies that are involved with exchanging information with other agencies or that are involved in data warehousing efforts should use this document as a guide.
Distributed concurrency control
Distributed concurrency control is the concurrency control of a system distributed over a computer network (Bernstein et al. 1987, Weikum and Vossen 2001). In database systems and transaction processing (transaction management) distributed concurrency control refers primarily to the concurrency control of a distributed database. It also refers to the concurrency control in a multidatabase (and other multi-transactional object) environment (e.g., federated database, grid computing, and cloud computing environments. A major goal for distributed concurrency control is distributed serializability (or global serializability for multidatabase systems). Distributed concurrency control poses special challenges beyond centralized one, primarily due to communication and computer latency. It often requires special techniques, like distributed lock manager over fast computer networks with low latency, like switched fabric (e.g., InfiniBand). The most common distributed concurrency control technique is strong strict two-phase locking (SS2PL, also named rigorousness), which is also a common centralized concurrency control technique. SS2PL provides both the serializability and strictness. Strictness, a special case of recoverability, is utilized for effective recovery from failure. For large-scale distribution and complex transactions, distributed locking's typical heavy performance penalty (due to delays, latency) can be saved by using the atomic commitment protocol, which is needed in a distributed database for (distributed) transactions' atomicity.
TikTokification
TikTokification (also written TikTok-ification) is a term used to describe the widespread adoption of TikTok's short-form, vertical video format and its algorithmic content-delivery model across the broader social media landscape. The phenomenon encompasses the strategic and cultural changes made by competing platforms such as Instagram, YouTube, Facebook, Snapchat, and LinkedIn in response to TikTok's global dominance. Beyond platform design, the term is also used more broadly to describe shifts in media consumption habits, advertising strategies, and, more critically, the potential cognitive and psychological effects associated with constant short-form video consumption. == Background == === Origins of short-form video === The short-form video format predates TikTok. Vine, launched in 2013, popularised six-second looping videos before shutting down in 2017. TikTok itself, known as Douyin in the Chinese market, was created by the Chinese technology company ByteDance in September 2016. Following its international expansion and its 2018 merger with Musical.ly, TikTok grew rapidly. By 2020, the application had surpassed two billion total downloads worldwide, with over 800 million monthly active users. A key driver of TikTok's success was its recommendation algorithm. The platform's "For You Page" (FYP) serves content to users based on behaviour rather than follower count, making it possible for unknown creators to achieve widespread reach organically. Analysts noted that TikTok serves "fast, visually engaging, and authentic videos that feel more like entertainment than advertising," fundamentally reshaping consumer expectations of digital content. TikTok has been described as "the center of the internet for young people," where users go for entertainment, news, trends, and shopping. As of the mid-2020s, TikTok had approximately 1.12 billion monthly active users. == Platform responses == TikTok's success compelled nearly every major social media platform to restructure its product around short-form video. In 2020, Instagram launched Reels and YouTube launched Shorts, both directly in response to TikTok's growth. Platforms like Meta's Instagram Reels and Google's YouTube Shorts subsequently expanded aggressively, launching new features, creator tools, and even considering separate standalone applications to compete. LinkedIn, traditionally a professional networking site, began experimenting with TikTok-style short-form vertical video feeds. Facebook launched a singular unified video feed combining Reels, long videos, and live videos, similar in structure to TikTok's feed. Snapchat redesigned its application to combine Stories and Spotlight into a unified entertainment feed. YouTube extended its Shorts format to allow videos up to three minutes in length, up from the previous limit of sixty seconds, as of October 2024. Despite these adaptations, experts noted that none of TikTok's rivals had matched its algorithmic precision as of mid-2025. == Societal and cultural impact == === Media and journalism === News organisations have also been affected by TikTokification. Short-form video grew rapidly as a format for news content, driven in large part by TikTok's popularity. According to Pew Research Center, 17% of adults in the United States reported regularly getting news from TikTok in 2024, with 63% of teenagers saying they used the platform as a news source. In response, major publishers began creating bespoke short-form content for TikTok's audience, with organisations such as the BBC building dedicated internal TikTok teams. === Advertising and commerce === TikTokification has had significant effects on the advertising industry. US social video advertising spending was projected to surpass linear television advertising spending for the first time in 2025. Global social commerce sales were projected to reach approximately $900 billion in 2025, with platforms like Douyin and TikTok driving a large share of that growth. TikTok itself generated an estimated $23.6 billion in advertising revenue in 2024. Short-form video has been described as bridging the gap between brand awareness and direct conversion. Surveys have found that consumers trust user-generated content 8.7 times more than influencer content and 6.6 times more than branded content, prompting brands to favour creator-led video formats. === Attention spans and cognitive effects === A growing body of research has examined the cognitive consequences of heavy short-form video consumption, a set of effects sometimes referred to as "TikTok Brain." A large systematic review and meta-analysis published in Psychological Bulletin, analysing data from 98,299 participants across 71 studies, found that the more short-form video content a person watches, the poorer their cognitive performance in attention and inhibitory control. The review also found that greater engagement with short-form video was associated with higher levels of anxiety, depression, and stress, as well as sleep disturbances. The platform's inherent demand for engaging content has resulted in the proliferation of sludge content, a genre of split screen video with the main video on the top and an unrelated attention-grabbing video on the bottom, typically repetitive gameplay (notably of the endless runner mobile game Subway Surfers) or oddly satisfying videos, designed to maximize viewer retention in cases where the main video may appear uninteresting and would normally cause the viewer to skip it. Sludge content is often described as overstimulating, reflecting and contributing to declining attention spans, though the scholarly evidence supporting such claims is not conclusive. Dr. Yann Poncin, associate professor at the Child Study Center at Yale University, noted that "infinite scrolling and short-form video are designed to capture your attention in short bursts," contrasting this with earlier entertainment formats that guided audiences through longer narratives. Research suggests that children and teenagers may be particularly vulnerable, with early exposure to rapid frame changes potentially conditioning the brain's neural pathways to require constant stimulation, making it more challenging to engage with slower-paced activities. A separate study published in Nature Communications by researchers at the Technical University of Denmark documented a notable decrease in collective attention span over time, attributing it in part to the increasing volume and pace of content production and consumption online. Researchers caution, however, that the majority of relevant studies are cross-sectional, meaning they capture data at a single point in time and cannot establish causality. It remains possible that individuals with pre-existing conditions such as anxiety or attention deficits may be more likely to engage heavily with these platforms as a coping mechanism. === Academic and sociological analysis === Scholars have framed TikTokification within the context of the attention economy. A 2024 academic analysis described TikTok as representing "a new paradigm of social media communication" shaped by youth culture, mobile technology, and the economics of attention, in which spectators become active contributors to a shared content pipeline. The same analysis noted that TikTok "reflects a new mode of communication influenced by avant-garde cinema, the use of mobile technology, and the social habits of particular social groups." US social media users were projected to spend 61.1% of their time on social networks watching videos in 2025, up from 33.3% in 2019, before TikTok became widely popular, underscoring the scale of the behavioural shift. == Monetisation challenges == Despite high engagement levels, monetising short-form video has remained difficult for platforms and creators alike. Unlike long-form YouTube content, short clips offer limited space for advertisers to insert advertisements. YouTube Shorts pays approximately four cents per 1,000 views, considerably less than its long-form counterpart. From 2025 onward, platforms began introducing creator funds, advertisements, and AI-driven content recommendations as part of broader efforts to make short-form video economically sustainable for creators.