TensorFlow Hub (also styled TF Hub) is an open-source machine learning library and online repository that provides TensorFlow model components, called modules. It is maintained by Google as part of the TensorFlow ecosystem and allows developers to discover, publish, and reuse pretrained models for tasks such as computer vision, natural language processing, and transfer learning. == Overview == TensorFlow Hub provides a central platform where developers and researchers can access pre-trained models and integrate them directly into TensorFlow workflows. Each module encapsulates a computation graph and its trained weights, with standardized input and output signatures. Modules can be loaded using the hub.load() function or through Keras integration via hub.KerasLayer, enabling users to perform transfer learning or feature extraction. == History == TensorFlow Hub was announced by Google in March 2018, with the first public version released shortly after. Its introduction coincided with the growing adoption of transfer learning techniques and the need for standardized model packaging. Over time, the hub expanded to include models such as the BERT family, MobileNet, EfficientNet, and the Universal Sentence Encoder. In 2020, research on “Regret selection in TensorFlow Hub” explored the problem of identifying optimal models for downstream tasks given a large repository of alternatives. == Applications == TensorFlow Hub hosts a variety of models across machine learning domains: Natural language processing: BERT, ALBERT language model, and Universal Sentence Encoder. Computer vision: ResNet, Inception (deep learning), MobileNet, EfficientNet. Speech and audio: spectrogram feature extractors and automatic speech recognition models. Multilingual embeddings: cross-lingual and sentence-level representations for machine translation and semantic similarity. Modules are widely used in education, academic research, and industry for prototyping and production deployment.
Dropbox Carousel
Dropbox Carousel was a photo and video management app offered by Dropbox. The third-party native app, available on Android and iOS, allowed users to store, manage, and organize photos. Photos were organized by date, time and event and backed up on Dropbox. It competed in this space against other online photo storage services such as Google's Google Photos, Apple's iCloud, and Yahoo's Flickr. Chris Lee, Dropbox's head of product development for Carousel described the app as an add-on to Dropbox, a “dedicated experience for photos and videos” and a space for “reliving personal memories”. == History == Mailbox founder, Gentry Underwood unveiled Carousel at a gathering in San Francisco on April 9, 2014. Much of the features in Carousel come from Snapjoy, a photo start-up, that Dropbox acquired on December 19, 2012. When Carousel was launched, it marked amongst many others, a series of acquisitions made by Dropbox to prep up before opening its stock for public offering. The acquisitions would help demonstrate its expansive product offerings pitching potential profitability to investors. In December 2015, Dropbox announced that Carousel would be shut down and some Carousel features would be integrated into the primary Dropbox application. On March 31, 2016, Carousel was deactivated. == Features == Carousel prompted users to free local storage once it had synced and backed-up local photos to the cloud. Flashback was a feature (enabled by default) that showed past photos or videos taken the same day, a year, or some years back. Flashback used an algorithm designed to identify human faces - resulting in greater likelihood of the user's picture or people in the user's close circle appearing. A scrollable timeline, which was earlier a scroll wheel, at the bottom let the user scroll to photo(s) at a specific date with a finger swipe.
Structured prediction
Structured prediction or structured output learning is an umbrella term for supervised machine learning techniques that involves predicting structured objects, rather than discrete or real values. Similar to commonly used supervised learning techniques, structured prediction models are typically trained by means of observed data in which the predicted value is compared to the ground truth, and this is used to adjust the model parameters. Due to the complexity of the model and the interrelations of predicted variables, the processes of model training and inference are often computationally infeasible, so approximate inference and learning methods are used. == Applications == An example application is the problem of translating a natural language sentence into a syntactic representation such as a parse tree. This can be seen as a structured prediction problem in which the structured output domain is the set of all possible parse trees. Structured prediction is used in a wide variety of domains including bioinformatics, natural language processing (NLP), speech recognition, and computer vision. === Example: sequence tagging === Sequence tagging is a class of problems prevalent in NLP in which input data are often sequential, for instance sentences of text. The sequence tagging problem appears in several guises, such as part-of-speech tagging (POS tagging) and named entity recognition. In POS tagging, for example, each word in a sequence must be 'tagged' with a class label representing the type of word: The main challenge of this problem is to resolve ambiguity: in the above example, the words "sentence" and "tagged" in English can also be verbs. While this problem can be solved by simply performing classification of individual tokens, this approach does not take into account the empirical fact that tags do not occur independently; instead, each tag displays a strong conditional dependence on the tag of the previous word. This fact can be exploited in a sequence model such as a hidden Markov model or conditional random field that predicts the entire tag sequence for a sentence (rather than just individual tags) via the Viterbi algorithm. == Techniques == Probabilistic graphical models form a large class of structured prediction models. In particular, Bayesian networks and random fields are popular. Other algorithms and models for structured prediction include inductive logic programming, case-based reasoning, structured SVMs, Markov logic networks, Probabilistic Soft Logic, and constrained conditional models. The main techniques are: Conditional random fields Structured support vector machines Structured k-nearest neighbours Recurrent neural networks, in particular Elman networks Transformers. === Structured perceptron === One of the easiest ways to understand algorithms for general structured prediction is the structured perceptron by Collins. This algorithm combines the perceptron algorithm for learning linear classifiers with an inference algorithm (classically the Viterbi algorithm when used on sequence data) and can be described abstractly as follows: First, define a function ϕ ( x , y ) {\displaystyle \phi (x,y)} that maps a training sample x {\displaystyle x} and a candidate prediction y {\displaystyle y} to a vector of length n {\displaystyle n} ( x {\displaystyle x} and y {\displaystyle y} may have any structure; n {\displaystyle n} is problem-dependent, but must be fixed for each model). Let G E N {\displaystyle GEN} be a function that generates candidate predictions. Then: Let w {\displaystyle w} be a weight vector of length n {\displaystyle n} For a predetermined number of iterations: For each sample x {\displaystyle x} in the training set with true output t {\displaystyle t} : Make a prediction y ^ {\displaystyle {\hat {y}}} : y ^ = a r g m a x { y ∈ G E N ( x ) } ( w T , ϕ ( x , y ) ) {\displaystyle {\hat {y}}={\operatorname {arg\,max} }\,\{y\in GEN(x)\}\,(w^{T},\phi (x,y))} Update w {\displaystyle w} (from y ^ {\displaystyle {\hat {y}}} towards t {\displaystyle t} ): w = w + c ( − ϕ ( x , y ^ ) + ϕ ( x , t ) ) {\displaystyle w=w+c(-\phi (x,{\hat {y}})+\phi (x,t))} , where c {\displaystyle c} is the learning rate. In practice, finding the argmax over G E N ( x ) {\displaystyle {GEN}({x})} is done using an algorithm such as Viterbi or a max-sum, rather than an exhaustive search through an exponentially large set of candidates. The idea of learning is similar to that for multiclass perceptrons.
Deterministic finite automaton
In the theory of computation, a branch of theoretical computer science, a deterministic finite automaton (DFA)—also known as deterministic finite acceptor (DFA), deterministic finite-state machine (DFSM), or deterministic finite-state automaton (DFSA)—is a finite-state machine that accepts or rejects a given string of symbols, by running through a state sequence uniquely determined by the string. Deterministic refers to the uniqueness of the computation run. In search of the simplest models to capture finite-state machines, Warren McCulloch and Walter Pitts were among the first researchers to introduce a concept similar to finite automata in 1943. The figure illustrates a deterministic finite automaton using a state diagram. In this example automaton, there are three states: S0, S1, and S2 (denoted graphically by circles). The automaton takes a finite sequence of 0s and 1s as input. For each state, there is a transition arrow leading out to a next state for both 0 and 1. Upon reading a symbol, a DFA jumps deterministically from one state to another by following the transition arrow. For example, if the automaton is currently in state S0 and the current input symbol is 1, then it deterministically jumps to state S1. A DFA has a start state (denoted graphically by an arrow coming in from nowhere) where computations begin, and a set of accept states (denoted graphically by a double circle) which help define when a computation is successful. A DFA is defined as an abstract mathematical concept, but is often implemented in hardware and software for solving various specific problems such as lexical analysis and pattern matching. For example, a DFA can model software that decides whether or not online user input such as email addresses are syntactically valid. DFAs have been generalized to nondeterministic finite automata (NFA) which may have several arrows of the same label starting from a state. Using the powerset construction method, every NFA can be translated to a DFA that recognizes the same language. DFAs, and NFAs as well, recognize exactly the set of regular languages. == Formal definition == A deterministic finite automaton M is a 5-tuple, (Q, Σ, δ, q0, F), consisting of a finite set of states Q a finite set of input symbols called the alphabet Σ a transition function δ : Q × Σ → Q an initial (or start) state q 0 ∈ Q {\displaystyle q_{0}\in Q} a set of accepting (or final) states F ⊆ Q {\displaystyle F\subseteq Q} Let w = a1a2...an be a string over the alphabet Σ. The automaton M accepts the string w if a sequence of states, r0, r1, ..., rn, exists in Q with the following conditions: r0 = q0 ri+1 = δ(ri, ai+1), for i = 0, ..., n − 1 r n ∈ F {\displaystyle r_{n}\in F} . In words, the first condition says that the machine starts in the start state q0. The second condition says that given each character of string w, the machine will transition from state to state according to the transition function δ. The last condition says that the machine accepts w if the last input of w causes the machine to halt in one of the accepting states. Otherwise, it is said that the automaton rejects the string. The set of strings that M accepts is the language recognized by M and this language is denoted by L(M). A deterministic finite automaton without accept states and without a starting state is known as a transition system or semiautomaton. For more comprehensive introduction of the formal definition see automata theory. == Example == The following example is of a DFA M, with a binary alphabet, which requires that the input contains an even number of 0s. M = (Q, Σ, δ, q0, F) where Q = {S1, S2} Σ = {0, 1} q0 = S1 F = {S1} and δ is defined by the following state transition table: The state S1 represents that there has been an even number of 0s in the input so far, while S2 signifies an odd number. A 1 in the input does not change the state of the automaton. When the input ends, the state will show whether the input contained an even number of 0s or not. If the input did contain an even number of 0s, M will finish in state S1, an accepting state, so the input string will be accepted. The language recognized by M is the regular language given by the regular expression (1) (0 (1) 0 (1)), where is the Kleene star, e.g., 1 denotes any number (possibly zero) of consecutive ones. == Variations == === Complete and incomplete === According to the above definition, deterministic finite automata are always complete: they define from each state a transition for each input symbol. While this is the most common definition, some authors use the term deterministic finite automaton for a slightly different notion: an automaton that defines at most one transition for each state and each input symbol; the transition function is allowed to be partial. When no transition is defined, such an automaton halts. === Local automata === A local automaton is a DFA, not necessarily complete, for which all edges with the same label lead to a single vertex. Local automata accept the class of local languages, those for which membership of a word in the language is determined by a "sliding window" of length two on the word. A Myhill graph over an alphabet A is a directed graph with vertex set A and subsets of vertices labelled "start" and "finish". The language accepted by a Myhill graph is the set of directed paths from a start vertex to a finish vertex: the graph thus acts as an automaton. The class of languages accepted by Myhill graphs is the class of local languages. === Randomness === When the start state and accept states are ignored, a DFA of n states and an alphabet of size k can be seen as a digraph of n vertices in which all vertices have k out-arcs labeled 1, ..., k (a k-out digraph). It is known that when k ≥ 2 is a fixed integer, with high probability, the largest strongly connected component (SCC) in such a k-out digraph chosen uniformly at random is of linear size and it can be reached by all vertices. It has also been proven that if k is allowed to increase as n increases, then the whole digraph has a phase transition for strong connectivity similar to Erdős–Rényi model for connectivity. In a random DFA, the maximum number of vertices reachable from one vertex is very close to the number of vertices in the largest SCC with high probability. This is also true for the largest induced sub-digraph of minimum in-degree one, which can be seen as a directed version of 1-core. == Closure properties == If DFAs recognize the languages that are obtained by applying an operation on the DFA recognizable languages then DFAs are said to be closed under the operation. The DFAs are closed under the following operations. For each operation, an optimal construction with respect to the number of states has been determined in state complexity research. Since DFAs are equivalent to nondeterministic finite automata (NFA), these closures may also be proved using closure properties of NFA. == As a transition monoid == A run of a given DFA can be seen as a sequence of compositions of a very general formulation of the transition function with itself. Here we construct that function. For a given input symbol a ∈ Σ {\displaystyle a\in \Sigma } , one may construct a transition function δ a : Q → Q {\displaystyle \delta _{a}:Q\rightarrow Q} by defining δ a ( q ) = δ ( q , a ) {\displaystyle \delta _{a}(q)=\delta (q,a)} for all q ∈ Q {\displaystyle q\in Q} . (This trick is called currying.) From this perspective, δ a {\displaystyle \delta _{a}} "acts" on a state in Q to yield another state. One may then consider the result of function composition repeatedly applied to the various functions δ a {\displaystyle \delta _{a}} , δ b {\displaystyle \delta _{b}} , and so on. Given a pair of letters a , b ∈ Σ {\displaystyle a,b\in \Sigma } , one may define a new function δ ^ a b = δ a ∘ δ b {\displaystyle {\widehat {\delta }}_{ab}=\delta _{a}\circ \delta _{b}} , where ∘ {\displaystyle \circ } denotes function composition. Clearly, this process may be recursively continued, giving the following recursive definition of δ ^ : Q × Σ ⋆ → Q {\displaystyle {\widehat {\delta }}:Q\times \Sigma ^{\star }\rightarrow Q} : δ ^ ( q , ϵ ) = q {\displaystyle {\widehat {\delta }}(q,\epsilon )=q} , where ϵ {\displaystyle \epsilon } is the empty string and δ ^ ( q , w a ) = δ a ( δ ^ ( q , w ) ) {\displaystyle {\widehat {\delta }}(q,wa)=\delta _{a}({\widehat {\delta }}(q,w))} , where w ∈ Σ ∗ , a ∈ Σ {\displaystyle w\in \Sigma ^{},a\in \Sigma } and q ∈ Q {\displaystyle q\in Q} . δ ^ {\displaystyle {\widehat {\delta }}} is defined for all words w ∈ Σ ∗ {\displaystyle w\in \Sigma ^{}} . A run of the DFA is a sequence of compositions of δ ^ {\displaystyle {\widehat {\delta }}} with itself. Repeated function composition forms a monoid. For the transition functions, this monoid is known as the transition monoid, or sometimes the transformation semigroup. The construction can also be reversed: given a δ ^ {\displaystyle {\wide
Language model
A language model is a computational model that predicts sequences in natural language. Language models are useful for a variety of tasks, including speech recognition, machine translation, natural language generation (generating more human-like text), optical character recognition, route optimization, handwriting recognition, grammar induction, information retrieval and disaster response. Large language models (LLMs), currently their most advanced form as of 2026, are predominantly based on transformers trained on larger datasets (frequently using texts scraped from the public internet). They have superseded recurrent neural network-based models, which had previously superseded the purely statistical models, such as the word n-gram language model. == History == Noam Chomsky did pioneering work on language models in the 1950s by developing a theory of formal grammars. In 1980, statistical approaches were explored and found to be more useful for many purposes than rule-based formal grammars. Discrete representations like word n-gram language models, with probabilities for discrete combinations of words, made significant advances. In the 2000s, continuous representations for words, such as word embeddings, began to replace discrete representations. Typically, the representation is a real-valued vector that encodes a word’s meaning such that words closer in vector space are similar in meaning and common relationships between words, such as plurality or gender, are preserved. == Pure statistical models == In 1980, the first significant statistical language model was proposed, and during the decade IBM performed 'Shannon-style' experiments, in which potential sources for language modeling improvement were identified by observing and analyzing the performance of human subjects in predicting or correcting text. === Models based on word n-grams === === Exponential === Maximum entropy language models encode the relationship between a word and the n-gram history using feature functions. The equation is P ( w m ∣ w 1 , … , w m − 1 ) = 1 Z ( w 1 , … , w m − 1 ) exp ( a T f ( w 1 , … , w m ) ) {\displaystyle P(w_{m}\mid w_{1},\ldots ,w_{m-1})={\frac {1}{Z(w_{1},\ldots ,w_{m-1})}}\exp(a^{T}f(w_{1},\ldots ,w_{m}))} where Z ( w 1 , … , w m − 1 ) {\displaystyle Z(w_{1},\ldots ,w_{m-1})} is the partition function, a {\displaystyle a} is the parameter vector, and f ( w 1 , … , w m ) {\displaystyle f(w_{1},\ldots ,w_{m})} is the feature function. In the simplest case, the feature function is just an indicator of the presence of a certain n-gram. It is helpful to use a prior on a {\displaystyle a} or some form of regularization. The log-bilinear model is another example of an exponential language model. === Skip-gram model === == Neural models == === Recurrent neural network === Continuous representations or embeddings of words are produced in recurrent neural network-based language models (known also as continuous space language models). Such continuous space embeddings help to alleviate the curse of dimensionality, which is the consequence of the number of possible sequences of words increasing exponentially with the size of the vocabulary, further causing a data sparsity problem. Neural networks avoid this problem by representing words as non-linear combinations of weights in a neural net. === Large language models === Although sometimes matching human performance, it is not clear whether they are plausible cognitive models. At least for recurrent neural networks, it has been shown that they sometimes learn patterns that humans do not, but fail to learn patterns that humans typically do. == Evaluation and benchmarks == Evaluation of the quality of language models is mostly done by comparison to human created sample benchmarks created from typical language-oriented tasks. Other, less established, quality tests examine the intrinsic character of a language model or compare two such models. Since language models are typically intended to be dynamic and to learn from data they see, some proposed models investigate the rate of learning, e.g., through inspection of learning curves. Various data sets have been developed for use in evaluating language processing systems. These include: Massive Multitask Language Understanding (MMLU) Corpus of Linguistic Acceptability GLUE benchmark Microsoft Research Paraphrase Corpus Multi-Genre Natural Language Inference Question Natural Language Inference Quora Question Pairs Recognizing Textual Entailment Semantic Textual Similarity Benchmark SQuAD question answering Test Stanford Sentiment Treebank Winograd NLI BoolQ, PIQA, SIQA, HellaSwag, WinoGrande, ARC, OpenBookQA, NaturalQuestions, TriviaQA, RACE, BIG-bench hard, GSM8k, RealToxicityPrompts, WinoGender, CrowS-Pairs
MultiValue database
A MultiValue database is a type of NoSQL and multidimensional database. It is typically considered synonymous with PICK, a database originally developed as the Pick operating system. MultiValue databases include commercial products from Rocket Software, Revelation, InterSystems, Northgate Information Solutions, ONgroup, and other companies. These databases differ from a relational database in that they have features that support and encourage the use of attributes which can take a list of values, rather than all attributes being single-valued. They are often categorized with MUMPS within the category of post-relational databases, although the data model actually pre-dates the relational model. Unlike SQL-DBMS tools, most MultiValue databases can be accessed both with or without SQL. == History == Don Nelson designed the MultiValue data model in the early to mid-1960s. Dick Pick, a developer at TRW, worked on the first implementation of this model for the US Army in 1965. Pick considered the software to be in the public domain because it was written for the military, this was but the first dispute regarding MultiValue databases that was addressed by the courts. Ken Simms wrote DataBASIC, sometimes known as S-BASIC, in the mid-1970s. It was based on Dartmouth BASIC, but had enhanced features for data management. Simms played a lot of Star Trek (a text-based early computer game originally written in Dartmouth BASIC) while developing the language, to ensure that DataBASIC functioned to his satisfaction. Three of the implementations of MultiValue - PICK version R77, Microdata Reality 3.x, and Prime Information 1.0 - were very similar. In spite of attempts to standardize, particularly by International Spectrum and the Spectrum Manufacturers Association, who designed a logo for all to use, there are no standards across MultiValue implementations. Subsequently, these flavors diverged, although with some cross-over. These streams of MultiValue database development could be classified as one stemming from PICK R83, one from Microdata Reality, and one from Prime Information. Because of the differences, some implementations have provisions for supporting several flavors of the languages. An attempt to document the similarities and differences can be found at the Post-Relational Database Reference (PRDB). One reasonable hypothesis for this data model lasting 50 years, with new database implementations of the model even in the 21st century is that it provides inexpensive database solutions. == Data model example == In a MultiValue database system: a database or schema is called an "account" a table or collection is called a "file" a column or field is called a field or an "attribute", which is composed of "multi-value attributes" and "sub-value attributes" to store multiple values in the same attribute. a row or document is called a "record" or "item" Data is stored using two separate files: a "file" to store raw data and a "dictionary" to store the format for displaying the raw data. For example, assume there's a file (table) called "PERSON". In this file, there is an attribute called "eMailAddress". The eMailAddress field can store a variable number of email address values in a single record. The list [[email protected], [email protected], [email protected]] can be stored and accessed via a single query when accessing the associated record. Achieving the same (one-to-many) relationship within a traditional relational database system would include creating an additional table to store the variable number of email addresses associated with a single "PERSON" record. However, modern relational database systems support this multi-value data model too. For example, in PostgreSQL, a column can be an array of any base type. == MultiValue Basic Language == Multivalue Basic (now commonly styled as mvBasic) is a family of programming languages more or less common (and portable) to all the multivalue databases derived from the original Pick Operating System. The variations between implementations are known as flavours. The language originates from Dartmouth Basic and the earliest implementation of PickBASIC (now D3 FlashBasic). Over time various customisations and extensions have been added to take advantage of capabilities added to the different flavours while staying mainly in sync. mvBasic statements and functions are designed to access and take advantage of the multivalue database model and providing the usual capabilities of most modern languages. For example, cryptography and communications. mvBasic is typeless and lends itself to structured programming techniques. Example code is available but limited. Whilst there are commercial applications and tools available, the multivalue database community has not embraced the open source library/package model to the degree seen with other languages. The typical mvBasic compiler compiles program source to a P-code executable object and runs in an interpreter, with D3 FlashBasic and jBASE being notable exceptions. == MultiValue Query Language == Known as ENGLISH, ACCESS, AQL, UniQuery, Retrieve, CMQL, and by many other names over the years, corresponding to the different MultiValue implementations, the MultiValue query language differs from SQL in several respects. Each query is issued against a single dictionary within the schema, which could be understood as a virtual file or a portal to the database through which to view the data. LIST PEOPLE LAST_NAME FIRST_NAME EMAIL_ADDRESSES WITH LAST_NAME LIKE "Van..." The above statement would list all e-mail addresses for each person whose last name starts with "Van". A single entry would be output for each person, with multiple lines showing the multiple e-mail addresses (without repeating other data about the person).
Kurt Keutzer
Kurt Keutzer (born November 9, 1955) is an American computer scientist. == Early life and education == Kurt Keutzer grew up in Indianapolis, Indiana. He earned a bachelor's degree in mathematics from Maharishi University of Management (formerly Mararishi International University) in 1978, and a PhD in computer science from Indiana University Bloomington in 1984. == Career == Keutzer joined Bell Labs in 1984, where he worked on logic synthesis. In 1991, he joined the electronic design automation company Synopsys, where he was promoted to chief technology officer. He subsequently joined the University of California, Berkeley as a professor in 1998. His research at Berkeley has focused on the intersection of high performance computing and machine learning. Working with a number of graduate students at Berkeley, Keutzer developed FireCaffe, which scaled the training of deep neural networks to over 100 GPUs. Later, with LARS and LAMB optimizers, they scaled it to over 1000 servers. Keutzer and his students also developed deep neural networks such as SqueezeNet, SqueezeDet, and SqueezeSeg, which can run efficiently on mobile devices. Keutzer co-founded DeepScale with his PhD student Forrest Iandola in 2015, and Keutzer served as the company's chief strategy officer. The firm was focused on developing deep neural networks for advanced driver assistance systems in passenger cars. On October 1, 2019, electric vehicle manufacturer Tesla, Inc. purchased DeepScale to augment and accelerate its self-driving vehicle work. == Honors and awards == Keutzer was named a Fellow of the IEEE in 1996. Recipient of DAC Most Influential Paper (MIP) award (24th DAC, 1987) for his "Dagon: technology binding and local optimization by DAG matching” publication. == Books by Keutzer == 1988. Dwight Hill, Don Shugard, John Fishburn, and Kurt Keutzer. Algorithms and Techniques for VLSI Layout Synthesis. Springer. 1994. Srinivas Devadas, Abhijit Ghosh, and Kurt Keutzer. Logic Synthesis. McGraw-Hill. 2002. David Chinnery and Kurt Keutzer. Closing the Gap Between ASIC & Custom: Tools and Techniques for High-Performance ASIC Design. Springer. (2nd edition appeared in 2007.) 2004. Pinhong Chen, Desmond A. Kirkpatrick, and Kurt Keutzer. Static Crosstalk-Noise Analysis: For Deep Sub-Micron Digital Designs. Springer. 2005. Matthias Gries and Kurt Keutzer. Building ASIPs: The Mescal Methodology. Springer.