A fuzzy concept is an idea of which the boundaries of application can vary considerably according to context or conditions, instead of being fixed once and for all. That means the idea is somewhat vague or imprecise. Yet it is not unclear or meaningless. It has a definite meaning, which can often be made more exact with further elaboration and specification — including a closer definition of the context in which the concept is used. The inverse of a "fuzzy concept" is a "crisp concept" (i.e. a precise concept). Fuzzy concepts are often used to navigate imprecision in the real world, when precise information is not available and an approximate indication is sufficient to be helpful. Although the linguist George Philip Lakoff already defined the semantics of a fuzzy concept in 1973 (inspired by an unpublished 1971 paper by Eleanor Rosch,) the term "fuzzy concept" rarely received a standalone entry in dictionaries, handbooks and encyclopedias. Sometimes it was defined in encyclopedia articles on fuzzy logic, or it was simply equated with a mathematical “fuzzy set”. A fuzzy concept can be "fuzzy" for many different reasons in different contexts. This makes it harder to provide a precise definition that covers all cases. Paradoxically, the definition of fuzzy concepts may itself be somewhat "fuzzy". Lotfi A. Zadeh, known as "the father of fuzzy logic", claimed that "vagueness connotes insufficient specificity, whereas fuzziness connotes unsharpness of class boundaries". Not all scholars agree. With increasing academic literature on the subject, the term "fuzzy concept" is now more widely recognized as a philosophical, linguistic or scientific category, and the study of the characteristics of fuzzy concepts and fuzzy language is known as fuzzy semantics. “Fuzzy logic” has become a generic term for many different kinds of many-valued logics, and is applied in many different areas of research, computer programming and industrial design. For engineers, "Fuzziness is imprecision or vagueness of definition." For computer scientists, a fuzzy concept is an idea which is "to an extent applicable" in a situation. It means that the concept can have gradations of significance or unsharp (variable) boundaries of application — a "fuzzy statement" is a statement which is true "to some extent", and that extent can often be represented by a scaled value (a score). For mathematicians, a "fuzzy concept" is usually a fuzzy set or a combination of such sets (see fuzzy mathematics and fuzzy set theory). In cognitive linguistics, the things that belong to a "fuzzy category" exhibit gradations of family resemblance, and the borders of the category are not clearly defined. Through most of the 20th century, the idea of reasoning with fuzzy concepts faced considerable resistance from Western academic elites. They did not want to endorse the use of imprecise concepts in research or argumentation, and they often regarded fuzzy logic with suspicion, derision or even hostility. That may partly explain why the idea of a "fuzzy concept" did not get a separate entry in encyclopedias, handbooks and dictionaries. Yet although people might not be aware of it, the use of fuzzy concepts has risen gigantically in all walks of life from the 1970s onward. That is mainly due to advances in electronic engineering, fuzzy mathematics and digital computer programming. The new technology allows very complex inferences about "variations on a theme" to be anticipated and fixed in a program. The Perseverance Mars rover, a driverless NASA vehicle used to explore the Jezero crater on the planet Mars, features fuzzy logic programming that steers it through rough terrain. Similarly, to the North, the Chinese Mars rover Zhurong used fuzzy logic algorithms to calculate its travel route in Utopia Planitia from sensor data. New neuro-fuzzy computational methods make it possible for machines to identify, measure, adjust and respond to fine gradations of significance with great precision. It means that practically useful concepts can be coded, sharply defined, and applied to all kinds of tasks, even if ordinarily these concepts are never exactly defined. Nowadays engineers, statisticians and programmers often represent fuzzy concepts mathematically, using fuzzy logic, fuzzy values, fuzzy variables and fuzzy sets (see also fuzzy set theory). Fuzzy logic is not "woolly thinking", but a "precise logic of imprecision" which reasons with graded concepts and gradations of truth. Fuzzy concepts and fuzzy logic often play a significant role in artificial intelligence programming, for example because they can model human cognitive processes more easily than other methods. == Origins == Vagueness and fuzziness have probably always been a part of human experience. In the West, ancient texts show that philosophers and scientists were already thinking critically about this in classical antiquity. Most often, they regarded vagueness as a problem: as an obstacle to clear thinking, as a source of confusion, or as an evasive tactic. It got in the way of providing clear orientation, guidance, direction and leadership. Therefore, vagueness became associated with a hermeneutic of suspicion — it was considered as something to avoid, as something undesirable. By contrast, in the ancient Chinese tradition of Daoist thought of Laozi and Zhuang Zhou, "vagueness is not regarded with suspicion, but is simply an acknowledged characteristic of the world around us" — a subject for meditation and a source of insight. === Sorites paradox === The ancient Sorites paradox raised the logical problem, of how we could exactly define the threshold at which a change in quantitative gradation turns into a qualitative or categorical difference. With some physical processes, this threshold seems relatively easy to identify. For example, water turns into steam at 100 °C or 212 °F. Of course, the boiling point depends partly on atmospheric pressure, which decreases at higher altitudes; it is also affected by the level of humidity — in that sense, the boiling point is "somewhat fuzzy", because it can vary under different conditions. Nevertheless, for every altitude, level of air pressure and degree of humidity, we can predict accurately what the boiling point will be, if we know the relevant conditions. With many other processes and gradations, however, the point of change is much more difficult to locate, and remains somewhat vague. Thus, the boundaries between qualitatively different things may be unsharp: we know that there are boundaries, but we cannot define them exactly. For example, to identify "the oldest city in the world", we have to define what counts as a city, and at what point a growing human settlement becomes a city. === The continuum fallacy and Loki's wager === According to the modern idea of the continuum fallacy, the fact that a statement is to an extent vague, does not automatically mean that it has no validity. The question then arises, of how (by what method or approach) we could ascertain and define the validity that the fuzzy statement does have. The Nordic myth of Loki's wager suggested that concepts that lack precise meanings or lack precise boundaries of application cannot be operated with, because they evade any clear definition. However, the 20th-century idea of "fuzzy concepts" proposes that "somewhat vague terms" can be operated with, because we can explicate and define the variability of their application — by assigning numbers to gradations of applicability. This idea sounds simple enough, but it had large implications. === Precursors and pioneers === In Western civilization, the intellectual recognition of fuzzy concepts has been traced back to a diversity of famous and less well-known thinkers, including (among many others) Eubulides, Epicurus, Plato, Cicero, William Ockham and John Buridan, Georg Wilhelm Friedrich Hegel, Karl Marx and Friedrich Engels, Friedrich Nietzsche, William James, Hugh MacColl, Charles S. Peirce, Hans Reichenbach, Carl Gustav Hempel, Max Black, Arto Salomaa, Ludwig Wittgenstein, Jan Łukasiewicz, Emil Leon Post, Alfred Tarski, Georg Cantor, Nicolai A. Vasiliev, Kurt Gödel, Stanisław Jaśkowski, Willard Van Orman Quine, George J. Klir, Petr Hájek, Joseph Goguen, Ronald R. Yager, Enrique Héctor Ruspini, Jan Pavelka, Didier Dubois, Bernadette Bouchon-Meunier, and Donald Knuth. Across at least two and a half millennia, all of them had something to say about graded concepts with unsharp boundaries. This suggests at least that the awareness of the existence of concepts with "fuzzy" characteristics, in one form or another, has a very long history in human thought. Quite a few 20th century logicians, mathematicians and philosophers also tried to analyze the characteristics of fuzzy concepts as a recognized species, sometimes with the aid of some kind of many-valued logic or substructural logic. An early attempt in the post-WW2 era to create a mathematical theory of sets with gradations of
Dropbox Carousel
Dropbox Carousel was a photo and video management app offered by Dropbox. The third-party native app, available on Android and iOS, allowed users to store, manage, and organize photos. Photos were organized by date, time and event and backed up on Dropbox. It competed in this space against other online photo storage services such as Google's Google Photos, Apple's iCloud, and Yahoo's Flickr. Chris Lee, Dropbox's head of product development for Carousel described the app as an add-on to Dropbox, a “dedicated experience for photos and videos” and a space for “reliving personal memories”. == History == Mailbox founder, Gentry Underwood unveiled Carousel at a gathering in San Francisco on April 9, 2014. Much of the features in Carousel come from Snapjoy, a photo start-up, that Dropbox acquired on December 19, 2012. When Carousel was launched, it marked amongst many others, a series of acquisitions made by Dropbox to prep up before opening its stock for public offering. The acquisitions would help demonstrate its expansive product offerings pitching potential profitability to investors. In December 2015, Dropbox announced that Carousel would be shut down and some Carousel features would be integrated into the primary Dropbox application. On March 31, 2016, Carousel was deactivated. == Features == Carousel prompted users to free local storage once it had synced and backed-up local photos to the cloud. Flashback was a feature (enabled by default) that showed past photos or videos taken the same day, a year, or some years back. Flashback used an algorithm designed to identify human faces - resulting in greater likelihood of the user's picture or people in the user's close circle appearing. A scrollable timeline, which was earlier a scroll wheel, at the bottom let the user scroll to photo(s) at a specific date with a finger swipe.
AMiner (database)
AMiner (formerly ArnetMiner) is a free online service used to index, search, and mine big scientific data. == Overview == AMiner (ArnetMiner) is designed to search and perform data mining operations against academic publications on the Internet, using social network analysis to identify connections between researchers, conferences, and publications. This allows it to provide services such as expert finding, geographic search, trend analysis, reviewer recommendation, association search, course search, academic performance evaluation, and topic modeling. AMiner was created as a research project in social influence analysis, social network ranking, and social network extraction. A number of peer-reviewed papers have been published arising from the development of the system. It has been in operation for more than three years, and has indexed 130,000,000 researchers and more than 265 million publications. The research was funded by the Chinese National High-tech R&D Program and the National Science Foundation of China. AMiner is commonly used in academia to identify relationships between and draw statistical correlations about research and researchers. It has attracted more than 10 million independent IP accesses from 220 countries and regions. The product has been used in Elsevier's SciVerse platform, and academic conferences such as SIGKDD, ICDM, PKDD, WSDM. == Operation == AMiner automatically extracts the researcher profile from the web. It collects and identifies the relevant pages, then uses a unified approach to extract data from the identified documents. It also extracts publications from online digital libraries using heuristic rules. It integrates the extracted researchers’ profiles and the extracted publications. It employs the researcher name as the identifier. A probabilistic framework has been proposed to deal with the name ambiguity problem in the integration. The integrated data is stored into a researcher network knowledge base (RNKB). The principal other product in the area are Google Scholar, Elsevier's Scirus, and the open source project CiteSeer. == History == It was initiated and created by professor Jie Tang from Tsinghua University, China. It was first launched in March 2006. The following provide a list of updates in the past years: March 2006, Version 0.1, Functions include researcher profiling, expert search, conference search, and publication search. The system was developed in Perl; August 2006, Version 1.0, The system was re-implemented in Java; July 2007, Version 2.0, New functions include researcher interest mining, association search, survey paper finding (unavailable now); April 2008, Version 3.0, New functions include query understanding, new GUI, and search log analysis; November 2008, Version 4.0, New functions include graph search, topic modeling, NSF/NSFC funding information extraction; April 2009, Version 5.0, New functions include Profile edition, open API service, Bole search, course search (unavailable now); December 2009, Version 6.0, New functions include academic performance evaluation, user feedback, conference analysis; May 2010, Version 7.0, New functions include name disambiguation, paper-reviewer recommendation, ArnetPage creation; March 2012, Version II, renamed as AMiner, rewrote all the codes and redesign the GUI. New functions include: geographic search, ArnetAPP platform. June 2014, Version II, renamed as AMiner, rewrote all the codes and redesign the GUI. New functions include: geographic search, ArnetAPP platform. December 2015, a completely new version got online. May 2017, professional version got online. April 2018, New functions include Trend Analysis, a deep learning based Name Disambiguation == Resources == AMiner published several datasets for academic research purpose, including Open Academic Graph, DBLP+citation (a data set augmenting citations into the DBLP data from Digital Bibliography & Library Project), Name Disambiguation, Social Tie Analysis. For more available datasets and source codes for research, please refer to.
Digital journalism
Digital journalism, also known as netizen journalism or online journalism, is a contemporary form of journalism where editorial content is distributed via the Internet, as opposed to publishing via print or broadcast. What constitutes digital journalism is debated amongst scholars. However, the primary product of journalism, which is news and features on current affairs, is presented solely or in combination as text, audio, video, or some interactive forms like storytelling stories or newsgames and disseminated through digital media technology. Fewer barriers to entry, lowered distribution costs and diverse computer networking technologies have led to the widespread practice of digital journalism. It has democratized the flow of information that was previously controlled by traditional media including newspapers, magazines, radio and television. Most readers expect online journalists to be reliable and competent, but these journalists often fail to meet this standard because they have very short deadlines and do not have enough resources to produce decent work. Some have asserted that a greater degree of creativity can be exercised with digital journalism when compared to traditional journalism and traditional media. The digital aspect may be central to the journalistic message and remains, to some extent, within the creative control of the writer, editor and/or publisher. It has been acknowledged that reports of its growth have tended to be exaggerated. In fact, a 2019 Pew survey showed a 16% decline in the time spent on online news sites since 2016. In the United States, reports issued by the Federal Communications Commission (FCC) in 2011 and by the Government Accountability Office (GAO) and the Congressional Research Service (CRS) in 2023 found that increases in newsroom staffing at digital-native news websites from 2008 to 2020 were not offsetting cuts in newsroom staffing among newspapers (which numbered in the tens of thousands of jobs), and that newspapers and television (which had been seeing declining newsroom staffing alongside newspapers) still employed the majority of payrolled newsroom staff in the United States in 2022 while online-only news websites employed less than 10%. The GAO and CRS reports noted further that the reduction in subscription and advertising revenue for the U.S. newspaper industry from 2000 to 2020 that constituted the overwhelming majority of its inflation-adjusted total revenue was not being offset by digital circulation or online advertising despite almost two-thirds of U.S. advertising spending in total by 2020 being online. Also, while the FCC report noted that local television stations in the United States had become some of the largest providers of local news online, the FCC found in a 2021 working paper that inflation-adjusted advertising revenue for television stations fell nationally from 2010 to 2018. == Overview == Digital journalism flows as journalism flows and is difficult to pinpoint where it is and where it is going. In partnership with digital media, digital journalism uses facets of digital media to perform journalist tasks, for example, using the internet as a tool rather than a singular form of digital media. There is no absolute agreement as to what constitutes digital journalism. Mu Lin argues that, "Web and mobile platforms demand us to adopt a platform-free mindset for an all-inclusive production approach – create the [digital] contents first, then distribute via appropriate platforms." The repurposing of print content for an online audience is sufficient for some, while others require content created with the digital medium's unique features like hypertextuality. Fondevila Gascón adds multimedia and interactivity to complete the digital journalism essence. For Deuze, online journalism can be functionally differentiated from other kinds of journalism by its technological component which journalists have to consider when creating or displaying content. Digital journalistic work may range from purely editorial content like CNN (produced by professional journalists) online to public-connectivity websites like Slashdot (communication lacking formal barriers of entry). The difference of digital journalism from traditional journalism may be in its re-conceptualised role of the reporter in relation to audiences and news organizations. The expectations of society for instant information was important for the evolution of digital journalism. However, it is likely that the exact nature and roles of digital journalism will not be fully known for some time. Some researchers even argue that the free distribution of online content, online advertisement and the new way recipients use news could undermine the traditional business model of mass media distributors that is based on single-copy sales, subscriptions and the selling of advertisement space. == History == The first type of digital journalism, called teletext, was invented in the UK in 1970. Teletext is a system allowing viewers to choose which stories they wish to read and see it immediately. The information provided through teletext is brief and instant, similar to the information seen in digital journalism today. The information was broadcast between the frames of a television signal in what was called the vertical blanking interval or VBI. American journalist Hunter S. Thompson relied on early digital communication technology beginning by using a fax machine to report from the 1971 US presidential campaign trail as documented in his book Fear and Loathing on the Campaign Trail. After the invention of teletext was the invention of videotex, of which Prestel was the world's first system, launching commercially in 1979 with various British newspapers, such as the Financial Times lining up to deliver newspaper stories online through it. Videotex closed down in 1986 due to failing to meet end-user demand. American newspaper companies took notice of the new technology and created their own videotex systems, the largest and most ambitious being Viewtron, a service of Knight-Ridder launched in 1981. Others were Keycom in Chicago and Gateway in Los Angeles. All of them had closed by 1986. Next came computer Bulletin Board Systems. In the late 1980s and early 1990s, several smaller newspapers started online news services using BBS software and telephone modems. The first of these was the Albuquerque Tribune in 1989. Computer Gaming World in September 1992 broke the news of Electronic Arts' acquisition of Origin Systems on Prodigy, before its next issue went to press. Online news websites began to proliferate in the 1990s. An early adopter was The News & Observer in Raleigh, North Carolina which offered online news as Nando. Steve Yelvington wrote on the Poynter Institute website about Nando, owned by The N&O, by saying "Nando evolved into the first serious, professional news site on the World Wide Web". It originated in the early 1990s as "NandO Land". It is believed that a major increase in digital online journalism occurred around this time when the first commercial web browsers, Netscape Navigator (1994) and Internet Explorer (1995). By 1996, most news outlets had an online presence. Although journalistic content was repurposed from original text/video/audio sources without change in substance, it could be consumed in different ways because of its online form through toolbars, topically grouped content, and intertextual links. A twenty-four-hour news cycle and new ways of user-journalist interaction web boards were among the features unique to the digital format. Later, portals such as AOL and Yahoo! and their news aggregators (sites that collect and categorize links from news sources) led to news agencies such as The Associated Press to supplying digitally suited content for aggregation beyond the limit of what client news providers could use in the past. Also, Salon, was founded in 1995. In 2001, the American Journalism Review called Salon the Internet's "preeminent independent venue for journalism." In 2008, for the first time, more Americans reported getting their national and international news from the internet, rather than newspapers. Young people aged 18 to 29 now primarily get their news via the Internet, according to a Pew Research Center report. Audiences to news sites continued to grow due to the launch of new news sites, continued investment in news online by conventional news organizations, and the continued growth in internet audiences overall. Sixty-five percent of youth now primarily access the news online. Mainstream news sites are the most widespread form of online news media production. As of 2000, the vast majority of journalists in the Western world now use the internet regularly in their daily work. In addition to mainstream news sites, digital journalism is found in index and category sites (sites without much original content but multiple links to existing news sites), meta- and comment sites (sites about
InteLex Past Masters
InteLex Past Masters is a collection of full-text web-based scholarly editions of classic works in the humanities. InteLex Corporation was founded in 1989 by its current chief executive officer, Mark Rooks, to produce electronic versions of the works of the great philosophers, based on existing scholarly editions. The company is located in Charlottesville, Virginia. Its databases are marketed to academic institutions, with pricing based on the individual collections purchased. Content is provided in XML and searchable image format and is accessed through the InteLex Corporation website. In addition to philosophy, subject coverage includes religious studies, English literature, women's writing, social science, and history of science. InteLex databases are found in institutions in over 65 countries around the world.
Neural scaling law
In machine learning, a neural scaling law is an empirical scaling law that describes how neural network performance changes as key factors are scaled up or down. These factors typically include the number of parameters, training dataset size, and training cost. Some models also exhibit performance gains by scaling inference through increased test-time compute (TTC), extending neural scaling laws beyond training to the deployment phase. == Introduction == In general, a deep learning model can be characterized by four parameters: model size, training dataset size, training cost, and the post-training error rate (e.g., the test set error rate). Each of these variables can be defined as a real number, usually written as N , D , C , L {\displaystyle N,D,C,L} (respectively: parameter count, dataset size, computing cost, and loss). A neural scaling law is a theoretical or empirical statistical law between these parameters. There are also other parameters with other scaling laws. === Size of the model === In most cases, the model's size is simply the number of parameters. However, one complication arises with the use of sparse models, such as mixture-of-expert models. With sparse models, during inference, only a fraction of their parameters are used. In comparison, most other kinds of neural networks, such as transformer models, always use all their parameters during inference. === Size of the training dataset === The size of the training dataset is usually quantified by the number of data points within it. Larger training datasets are typically preferred, as they provide a richer and more diverse source of information from which the model can learn. This can lead to improved generalization performance when the model is applied to new, unseen data. However, increasing the size of the training dataset also increases the computational resources and time required for model training. With the "pretrain, then finetune" method used for most large language models, there are two kinds of training dataset: the pretraining dataset and the finetuning dataset. Their sizes have different effects on model performance. Generally, the finetuning dataset is less than 1% the size of pretraining dataset. In some cases, a small amount of high quality data suffices for finetuning, and more data does not necessarily improve performance. Many scaling laws, due to their inherent diminishing returns nature, value data based on a submodular set function which was shown in a paper on this topic. === Cost of training === Training cost is typically measured in terms of time (how long it takes to train the model) and computational resources (how much processing power and memory are required). It is important to note that the cost of training can be significantly reduced with efficient training algorithms, optimized software libraries, and parallel computing on specialized hardware such as GPUs or TPUs. The cost of training a neural network model is a function of several factors, including model size, training dataset size, the training algorithm complexity, and the computational resources available. In particular, doubling the training dataset size does not necessarily double the cost of training, because one may train the model for several times over the same dataset (each being an "epoch"). === Performance === The performance of a neural network model is evaluated based on its ability to accurately predict the output given some input data. Common metrics for evaluating model performance include: Negative log-likelihood per token (logarithm of perplexity) for language modeling; Accuracy, precision, recall, and F1 score for classification tasks; Mean squared error (MSE) or mean absolute error (MAE) for regression tasks; Elo rating in a competition against other models, such as gameplay or preference by a human judge. Performance can be improved by using more data, larger models, different training algorithms, regularizing the model to prevent overfitting, and early stopping using a validation set. When the performance is a number bounded within the range of [ 0 , 1 ] {\displaystyle [0,1]} , such as accuracy, precision, etc., it often scales as a sigmoid function of cost, as seen in the figures. == Examples == === (Hestness, Narang, et al, 2017) === The 2017 paper is a common reference point for neural scaling laws fitted by statistical analysis on experimental data. Previous works before the 2000s, as cited in the paper, were either theoretical or orders of magnitude smaller in scale. Whereas previous works generally found the scaling exponent to scale like L ∝ D − α {\displaystyle L\propto D^{-\alpha }} , with α ∈ { 0.5 , 1 , 2 } {\displaystyle \alpha \in \{0.5,1,2\}} , the paper found that α ∈ [ 0.07 , 0.35 ] {\displaystyle \alpha \in [0.07,0.35]} . Of the factors they varied, only task can change the exponent α {\displaystyle \alpha } . Changing the architecture optimizers, regularizers, and loss functions, would only change the proportionality factor, not the exponent. For example, for the same task, one architecture might have L = 1000 D − 0.3 {\displaystyle L=1000D^{-0.3}} while another might have L = 500 D − 0.3 {\displaystyle L=500D^{-0.3}} . They also found that for a given architecture, the number of parameters necessary to reach lowest levels of loss, given a fixed dataset size, grows like N ∝ D β {\displaystyle N\propto D^{\beta }} for another exponent β {\displaystyle \beta } . They studied machine translation with LSTM ( α ∼ 0.13 {\displaystyle \alpha \sim 0.13} ), generative language modelling with LSTM ( α ∈ [ 0.06 , 0.09 ] , β ≈ 0.7 {\displaystyle \alpha \in [0.06,0.09],\beta \approx 0.7} ), ImageNet classification with ResNet ( α ∈ [ 0.3 , 0.5 ] , β ≈ 0.6 {\displaystyle \alpha \in [0.3,0.5],\beta \approx 0.6} ), and speech recognition with two hybrid (LSTMs complemented by either CNNs or an attention decoder) architectures ( α ≈ 0.3 {\displaystyle \alpha \approx 0.3} ). === (Henighan, Kaplan, et al, 2020) === A 2020 analysis studied statistical relations between C , N , D , L {\displaystyle C,N,D,L} over a wide range of values and found similar scaling laws, over the range of N ∈ [ 10 3 , 10 9 ] {\displaystyle N\in [10^{3},10^{9}]} , C ∈ [ 10 12 , 10 21 ] {\displaystyle C\in [10^{12},10^{21}]} , and over multiple modalities (text, video, image, text to image, etc.). In particular, the scaling laws it found are (Table 1 of ): For each modality, they fixed one of the two C , N {\displaystyle C,N} , and varying the other one ( D {\displaystyle D} is varied along using D = C / 6 N {\displaystyle D=C/6N} ), the achievable test loss satisfies L = L 0 + ( x 0 x ) α {\displaystyle L=L_{0}+\left({\frac {x_{0}}{x}}\right)^{\alpha }} where x {\displaystyle x} is the varied variable, and L 0 , x 0 , α {\displaystyle L_{0},x_{0},\alpha } are parameters to be found by statistical fitting. The parameter α {\displaystyle \alpha } is the most important one. When N {\displaystyle N} is the varied variable, α {\displaystyle \alpha } ranges from 0.037 {\displaystyle 0.037} to 0.24 {\displaystyle 0.24} depending on the model modality. This corresponds to the α = 0.34 {\displaystyle \alpha =0.34} from the Chinchilla scaling paper. When C {\displaystyle C} is the varied variable, α {\displaystyle \alpha } ranges from 0.048 {\displaystyle 0.048} to 0.19 {\displaystyle 0.19} depending on the model modality. This corresponds to the β = 0.28 {\displaystyle \beta =0.28} from the Chinchilla scaling paper. Given fixed computing budget, optimal model parameter count is consistently around N o p t ( C ) = ( C 5 × 10 − 12 petaFLOP-day ) 0.7 = 9.0 × 10 − 7 C 0.7 {\displaystyle N_{opt}(C)=\left({\frac {C}{5\times 10^{-12}{\text{petaFLOP-day}}}}\right)^{0.7}=9.0\times 10^{-7}C^{0.7}} The parameter 9.0 × 10 − 7 {\displaystyle 9.0\times 10^{-7}} varies by a factor of up to 10 for different modalities. The exponent parameter 0.7 {\displaystyle 0.7} varies from 0.64 {\displaystyle 0.64} to 0.75 {\displaystyle 0.75} for different modalities. This exponent corresponds to the ≈ 0.5 {\displaystyle \approx 0.5} from the Chinchilla scaling paper. It's "strongly suggested" (but not statistically checked) that D o p t ( C ) ∝ N o p t ( C ) 0.4 ∝ C 0.28 {\displaystyle D_{opt}(C)\propto N_{opt}(C)^{0.4}\propto C^{0.28}} . This exponent corresponds to the ≈ 0.5 {\displaystyle \approx 0.5} from the Chinchilla scaling paper. The scaling law of L = L 0 + ( C 0 / C ) 0.048 {\displaystyle L=L_{0}+(C_{0}/C)^{0.048}} was confirmed during the training of GPT-3 (Figure 3.1 ). === Chinchilla scaling (Hoffmann, et al, 2022) === One particular scaling law ("Chinchilla scaling") states that, for a large language model (LLM) autoregressively trained for one epoch, with a cosine learning rate schedule, we have: { C = C 0 N D L = A N α + B D β + L 0 {\displaystyle {\begin{cases}C=C_{0}ND\\L={\frac {A}{N^{\alpha }}}+{\frac {B}{D^{\beta }}}+L_{0}\end{cases}}} where the variables are C {\displaystyle C} is the cost o
Modulation error ratio
The modulation error ratio (MER) is a measure used to quantify the performance of a digital radio (or digital TV) transmitter or receiver in a communications system using digital modulation (such as QAM). A signal sent by an ideal transmitter or received by a receiver would have all constellation points precisely at the ideal locations, however various imperfections in the implementation (such as noise, low image rejection ratio, phase noise, carrier suppression, distortion, etc.) or signal path cause the actual constellation points to deviate from the ideal locations. Transmitter MER can be measured by specialized equipment, which demodulates the received signal in a similar way to how a real radio demodulator does it. Demodulated and detected signal can be used as a reasonably reliable estimate for the ideal transmitted signal in MER calculation. == Definition == An error vector is a vector in the I-Q plane between the ideal constellation point and the point received by the receiver. The Euclidean distance between the two points is its magnitude. The modulation error ratio is equal to the ratio of the root mean square (RMS) power (in Watts) of the reference vector to the power (in Watts) of the error. It is defined in dB as: M E R ( d B ) = 10 log 10 ( P s i g n a l P e r r o r ) {\displaystyle \mathrm {MER(dB)} =10\log _{10}\left({P_{\mathrm {signal} } \over P_{\mathrm {error} }}\right)} where Perror is the RMS power of the error vector, and Psignal is the RMS power of ideal transmitted signal. MER is defined as a percentage in a compatible (but reciprocal) way: M E R ( % ) = P e r r o r P s i g n a l × 100 % {\displaystyle \mathrm {MER(\%)} ={\sqrt {P_{\mathrm {error} } \over P_{\mathrm {signal} }}}\times 100\%} with the same definitions. MER is closely related to error vector magnitude (EVM), but MER is calculated from the average power of the signal. MER is also closely related to signal-to-noise ratio. MER includes all imperfections including deterministic amplitude imbalance, quadrature error and distortion, while noise is random by nature.