In natural language processing, a sentence embedding is a representation of a sentence as a vector of numbers which encodes meaningful semantic information. State of the art embeddings are based on the learned hidden layer representation of dedicated sentence transformer models. BERT pioneered an approach involving the use of a dedicated [CLS] token prepended to the beginning of each sentence inputted into the model; the final hidden state vector of this token encodes information about the sentence and can be fine-tuned for use in sentence classification tasks. In practice however, BERT's sentence embedding with the [CLS] token achieves poor performance, often worse than simply averaging non-contextual word embeddings. SBERT later achieved superior sentence embedding performance by fine tuning BERT's [CLS] token embeddings through the usage of a siamese neural network architecture on the SNLI dataset. Other approaches are loosely based on the idea of distributional semantics applied to sentences. Skip-Thought trains an encoder-decoder structure for the task of neighboring sentences predictions; this has been shown to achieve worse performance than approaches such as InferSent or SBERT. An alternative direction is to aggregate word embeddings, such as those returned by Word2vec, into sentence embeddings. The most straightforward approach is to simply compute the average of word vectors, known as continuous bag-of-words (CBOW). However, more elaborate solutions based on word vector quantization have also been proposed. One such approach is the vector of locally aggregated word embeddings (VLAWE), which demonstrated performance improvements in downstream text classification tasks. == Applications == In recent years, sentence embedding has seen a growing level of interest due to its applications in natural language queryable knowledge bases through the usage of vector indexing for semantic search. LangChain for instance utilizes sentence transformers for purposes of indexing documents. In particular, an indexing is generated by generating embeddings for chunks of documents and storing (document chunk, embedding) tuples. Then given a query in natural language, the embedding for the query can be generated. A top k similarity search algorithm is then used between the query embedding and the document chunk embeddings to retrieve the most relevant document chunks as context information for question answering tasks. This approach is also known formally as retrieval-augmented generation. Though not as predominant as BERTScore, sentence embeddings are commonly used for sentence similarity evaluation which sees common use for the task of optimizing a Large language model's generation parameters is often performed via comparing candidate sentences against reference sentences. By using the cosine-similarity of the sentence embeddings of candidate and reference sentences as the evaluation function, a grid-search algorithm can be utilized to automate hyperparameter optimization. == Evaluation == A way of testing sentence encodings is to apply them on Sentences Involving Compositional Knowledge (SICK) corpus for both entailment (SICK-E) and relatedness (SICK-R). In the best results are obtained using a BiLSTM network trained on the Stanford Natural Language Inference (SNLI) Corpus. The Pearson correlation coefficient for SICK-R is 0.885 and the result for SICK-E is 86.3. A slight improvement over previous scores is presented in: SICK-R: 0.888 and SICK-E: 87.8 using a concatenation of bidirectional Gated recurrent unit.
Digital video effect
Digital video effects (DVEs) are visual effects that provide comprehensive live video image manipulation, in the same form as optical printer effects in film. DVEs differ from standard video switcher effects (often referred to as analog effects) such as wipes or dissolves, in that they deal primarily with resizing, distortion or movement of the image. Modern video switchers often contain internal DVE functionality. Modern DVE devices are incorporated in high-end broadcast video switchers. Early examples of DVE devices found in the broadcast post-production industry include the Ampex Digital Optics (ADO), Quantel DPE-5000, Vital Squeezoom, NEC E-Flex and the Abekas A5x series of DVEs. By 1988, Grass Valley Group caught up with the competition with their Kaleidoscope, which integrated ADO-type effects with their widely used line of broadcast switching gear. DVEs are used by the broadcast television industry in live television production environments like television studios and outside broadcasts. They are commonly used in video post-production.
Vidby
Vidby AG (stylized in lower-case) is a start-up based in Rotkreuz, Switzerland specializing in AI language translation for videos. Founded by Alexander Konovalov (uk:Олександр Коновалов) and Eugen von Rubinberg in September 2021, the company has especially garnered attention for its use in translating speeches given by President Volodymyr Zelenskyy during the Russian invasion of Ukraine. == History == Vidby AG was founded by Alexander Konovalov and Eugen von Rubinberg. Konovalov is a native of Ukraine and retains Ukrainian citizenship; Rubinberg came to Switzerland from Germany and holds German citizenship. Both are residents of Switzerland. The latter founded his first business, a trading company, at age 16. In 2013, the business partners launched a consumer-oriented video-call translation service called DROTR (Droid Translator) AG, utilizing a Konovalov-created AI-powered language translation technology enabling simultaneous translation of messages, voice and video calls in 104 languages (written), with 44 available in spoken form. This was the world's first video calling app with translation. The technology was pronounced a competitor of Skype and Viber by Forbes and claimed first prize at the "Innovative Breakthrough 2013" Competition. In 2021, with a new business-oriented focus, DROTR became Vidby, with the former Google technology partners Konovalov and Rubinberg remaining at the helm, each with the title Co-CEO. While headquartered in Switzerland, Vidby's development team is, according to the company's founders, based in Ukraine. The technology behind Vidby has an accuracy level variously reported as up to 99 percent or 99 to 100 percent, equalling the highest level of human translation. Additionally, the technology is capable of removing the original language while maintaining ambient sounds. Currently, some 70 languages plus 60 dialects are possible with the algorithm-based technology. == Notable use == In addition to its use with speeches delivered by Pope Francis, the technology has been provided to Ukrainian authorities and embassies during the ongoing military conflict with Russia free of remuneration. By July, 2022, some 70 speeches given by President Zelenskyy totalling 650 minutes had been translated into 30 languages, for a total of over 10,000 minutes of video material. Of its use in translating Zelenskyy's wartime speeches, Konovalov has said, "Like any citizen, I want to help defend my country." Notable corporate clients of Vidby include Samsung, Siemens, Cisco, Kärcher, Generali and McDonald's Corporation; an academic client is Harvard University. Google Cloud Technology Partner status of Vidby was confirmed officially after a six-month audit in December 2022. Denys Krasnikov, a Vidby co-founder, is responsible for cooperation with Google, YouTube, Microsoft, and other key partners. After the launch of multilingual YouTube channels, Vidby started AI translating and dubbing creators' videos for this new type of channel at the end of February 2023. == Accolades == Vidby headed a list of the five best video translation services as named by TechRadar Deutschland in September, 2022. In the same month, Tech Times named Vidby #1 in their list of the five best such services. It similarly topped a list of the five best content translation technologies as judged by European Business Review in October, 2022. Prior to these lead-position rankings (August, 2022), it was featured as Business Insider's special start-up recommendation (German: "Unser Lesetipp auf Gründerszene"). In 2023, YouTube recognized Vidby as its recommended vendor.
Noise-based logic
Noise-based logic (NBL) is a class of multivalued deterministic logic schemes, developed in the twenty-first century, where the logic values and bits are represented by different realizations of a stochastic process. The concept of noise-based logic and its name was created by Laszlo B. Kish. In its foundation paper it is noted that the idea was inspired by the stochasticity of brain signals and by the unconventional noise-based communication schemes, such as the Kish cypher. == The noise-based logic space and hyperspace == The logic values are represented by multi-dimensional "vectors" (orthogonal functions) and their superposition, where the orthogonal basis vectors are independent noises. By the proper combination (products or set-theoretical products) of basis-noises, which are called noise-bit, a logic hyperspace can be constructed with D(N) = 2N number of dimensions, where N is the number of noise-bits. Thus N noise-bits in a single wire correspond to a system of 2N classical bits that can express 22N different logic values. Independent realizations of a stochastic process of zero mean have zero cross-correlation with each other and with other stochastic processes of zero mean. Thus the basis noise vectors are orthogonal not only to each other but they and all the noise-based logic states (superpositions) are orthogonal also to any background noises in the hardware. Therefore, the noise-based logic concept is robust against background noises, which is a property that can potentially offer a high energy-efficiency. == The types of signals used in noise-based logic == In the paper, where noise-based logic was first introduced, generic stochastic-processes with zero mean were proposed and a system of orthogonal sinusoidal signals were also proposed as a deterministic-signal version of the logic system. The mathematical analysis about statistical errors and signal energy was limited to the cases of Gaussian noises and superpositions as logic signals in the basic logic space and their products and superpositions of their products in the logic hyperspace (see also. In the subsequent brain logic scheme, the logic signals were (similarly to neural signals) unipolar spike sequences generated by a Poisson process, and set-theoretical unifications (superpositions) and intersections (products) of different spike sequences. Later, in the instantaneous noise-based logic schemes and computation works, random telegraph waves (periodic time, bipolar, with fixed absolute value of amplitude) were also utilized as one of the simplest stochastic processes available for NBL. With choosing unit amplitude and symmetric probabilities, the resulting random-telegraph wave has 0.5 probability to be in the +1 or in the −1 state which is held over the whole clock period. == The noise-based logic gates == Noise-based logic gates can be classified according to the method the input identifies the logic value at the input. The first gates analyzed the statistical correlations between the input signal and the reference noises. The advantage of these is the robustness against background noise. The disadvantage is the slow speed and higher hardware complexity. The instantaneous logic gates are fast, they have low complexity but they are not robust against background noises. With either neural spike type signals or with bipolar random-telegraph waves of unity absolute amplitude, and randomness only in the sign of the amplitude offer very simple instantaneous logic gates. Then linear or analog devices unnecessary and the scheme can operate in the digital domain. However, whenever instantaneous logic must be interfaced with classical logic schemes, the interface must use correlator-based logic gates for an error-free signal. == Universality of noise-based logic == All the noise-based logic schemes listed above have been proven universal. The papers typically produce the NOT and the AND gates to prove universality, because having both of them is a satisfactory condition for the universality of a Boolean logic. == Computation by noise-based logic == The string verification work over a slow communication channel shows a powerful computing application where the methods is inherently based on calculating the hash function. The scheme is based on random telegraph waves and it is mentioned in the paper that the authors intuitively conclude that the intelligence of the brain is using similar operations to make a reasonably good decision based on a limited amount of information. The superposition of the first D(N) = 2N integer numbers can be produced with only 2N operations, which the authors call "Achilles ankle operation" in the paper. == Computer chip realization of noise-based logic == Preliminary schemes have already been published to utilize noise-based logic in practical computers. However, it is obvious from these papers that this young field has yet a long way to go before it will be seen in everyday applications.
Stanhope Demonstrator
The Stanhope Demonstrator was the first machine to solve problems in logic. It was designed by Charles Stanhope, 3rd Earl Stanhope to demonstrate consequences in logic symbolically. The first model was constructed in 1775. It consisted of two slides coloured red and gray mounted in a square brass frame. This could be used to demonstrate the solution to a syllogistic type of problem in which objects might have two different properties and the question was how many would have both properties. Scales marked zero to ten were used to set the numbers or proportions of objects with the two properties. This form of inference anticipated the numerically definite syllogism which Augustus De Morgan laid out in his book, Formal Logic, in 1847. == Construction == The device was a brass plate about four inches square which was mounted on a piece of mahogany which was three-quarters of an inch thick. There was an opening with a depression in the wood about one and a half inches square and half an inch deep. This opening was called the holon, meaning "whole", and represented the full set of objects under consideration. A slide of red translucent glass could be inserted from the right across the holon. A slide of gray wood could be slid under the red slide. When the device was used for the "Rule for the Logic of Certainty", the gray slider was inserted from the left. When it was used for the "Rule for the Logic of Probability", the gray slider was inserted from above. The red and the gray sliders represented the two affirmative propositions which were being combined. Stanhope called these ho and los. At least four of the devices with this square style were built. In 1879, Robert Harley wrote that he had one which he had been given by Stanhope's great-grandson, Arthur, who had kept one. The other two were owned by Henry Prevost Babbage – the son of Charles Babbage, who continued his work on the Analytical Engine. One of the devices was donated to the Science Museum, London by the last Earl in 1953. Other styles, such as circular models, were constructed, but these were less convenient.
Spreading activation
Spreading activation is a method for searching associative networks, biological and artificial neural networks, or semantic networks. The search process is initiated by labeling a set of source nodes (e.g. concepts in a semantic network) with weights or "activation" and then iteratively propagating or "spreading" that activation out to other nodes linked to the source nodes. Most often these "weights" are real values that decay as activation propagates through the network. When the weights are discrete this process is often referred to as marker passing. Activation may originate from alternate paths, identified by distinct markers, and terminate when two alternate paths reach the same node. However brain studies show that several different brain areas play an important role in semantic processing. Spreading activation in semantic networks as a model were invented in cognitive psychology to model the fan out effect. Spreading activation can also be applied in information retrieval, by means of a network of nodes representing documents and terms contained in those documents. == Cognitive psychology == As it relates to cognitive psychology, spreading activation is the theory of how the brain iterates through a network of associated ideas to retrieve specific information. The spreading activation theory presents the array of concepts within our memory as cognitive units, each consisting of a node and its associated elements or characteristics, all connected together by edges. A spreading activation network can be represented schematically, in a sort of web diagram with shorter lines between two nodes meaning the ideas are more closely related and will typically be associated more quickly to the original concept. In memory psychology, the spreading activation model holds that people organize their knowledge of the world based on their personal experiences, which in turn form the network of ideas that is the person's knowledge of the world. When a word (the target) is preceded by an associated word (the prime) in word recognition tasks, participants seem to perform better in the amount of time that it takes them to respond. For instance, subjects respond faster to the word "doctor" when it is preceded by "nurse" than when it is preceded by an unrelated word like "carrot". This semantic priming effect with words that are close in meaning within the cognitive network has been seen in a wide range of tasks given by experimenters, ranging from sentence verification to lexical decision and naming. As another example, if the original concept is "red" and the concept "vehicles" is primed, they are much more likely to say "fire engine" instead of something unrelated to vehicles, such as "cherries". If instead "fruits" was primed, they would likely name "cherries" and continue on from there. The activation of pathways in the network has everything to do with how closely linked two concepts are by meaning, as well as how a subject is primed. == Algorithm == A directed graph is populated by Nodes[ 1...N ] each having an associated activation value A [ i ] which is a real number in the range [0.0 ... 1.0]. A Link[ i, j ] connects source node[ i ] with target node[ j ]. Each edge has an associated weight W [ i, j ] usually a real number in the range [0.0 ... 1.0]. Parameters: Firing threshold F, a real number in the range [0.0 ... 1.0] Decay factor D, a real number in the range [0.0 ... 1.0] Steps: Initialize the graph setting all activation values A [ i ] to zero. Set one or more origin nodes to an initial activation value greater than the firing threshold F. A typical initial value is 1.0. For each unfired node [ i ] in the graph having an activation value A [ i ] greater than the node firing threshold F: For each Link [ i, j ] connecting the source node [ i ] with target node [ j ], adjust A [ j ] = A [ j ] + (A [ i ] W [ i, j ] D) where D is the decay factor. If a target node receives an adjustment to its activation value so that it would exceed 1.0, then set its new activation value to 1.0. Likewise maintain 0.0 as a lower bound on the target node's activation value should it receive an adjustment to below 0.0. Once a node has fired it may not fire again, although variations of the basic algorithm permit repeated firings and loops through the graph. Nodes receiving a new activation value that exceeds the firing threshold F are marked for firing on the next spreading activation cycle. If activation originates from more than one node, a variation of the algorithm permits marker passing to distinguish the paths by which activation is spread over the graph The procedure terminates when either there are no more nodes to fire or in the case of marker passing from multiple origins, when a node is reached from more than one path. Variations of the algorithm that permit repeated node firings and activation loops in the graph, terminate after a steady activation state, with respect to some delta, is reached, or when a maximum number of iterations is exceeded. == Examples ==
Fuzzy Control Language
Fuzzy Control Language, or FCL, is a language for implementing fuzzy logic, especially fuzzy control. It was standardized by IEC 61131-7. It is a domain-specific programming language: it has no features unrelated to fuzzy logic, so it is impossible to even print "Hello, world!". Therefore, one does not write a program in FCL, but one may write part of it in FCL. == Example == RULE 0: IF (temperature IS cold) THEN (output IS low) RULE 1: IF (temperature IS very cold) THEN (output IS high) == Limitations == FCL is not an entirely complete fuzzy language, for instance, it does not support "hedges", which are adverbs that modify the set. For instance, the programmer cannot write: RULE 0: If (Temperature is VERY COLD) then (Output is VERY HIGH) However, the programmer can simply define new sets for "very cold" and "very high". FCL also lacks support for higher-order fuzzy sets, subsets, and so on. None of these features are essential to fuzzy control, although they may be nice to have.