Defuzzification is the process of producing a quantifiable result in crisp logic, given fuzzy sets and corresponding membership degrees. It is the process that maps a fuzzy set to a crisp set. It is typically needed in fuzzy control systems. These systems will have a number of rules that transform a number of variables into a fuzzy result, that is, the result is described in terms of membership in fuzzy sets. For example, rules designed to decide how much pressure to apply might result in "Decrease Pressure (15%), Maintain Pressure (34%), Increase Pressure (72%)". Defuzzification is interpreting the membership degrees of the fuzzy sets into a specific decision or real value. The simplest but least useful defuzzification method is to choose the set with the highest membership, in this case, "Increase Pressure" since it has a 72% membership, and ignore the others, and convert this 72% to some number. The problem with this approach is that it loses information. The rules that called for decreasing or maintaining pressure might as well have not been there in this case. A common and useful defuzzification technique is center of gravity. First, the results of the rules must be added together in some way. The most typical fuzzy set membership function has the graph of a triangle. Now, if this triangle were to be cut in a straight horizontal line somewhere between the top and the bottom, and the top portion were to be removed, the remaining portion forms a trapezoid. The first step of defuzzification typically "chops off" parts of the graphs to form trapezoids (or other shapes if the initial shapes were not triangles). For example, if the output has "Decrease Pressure (15%)", then this triangle will be cut 15% the way up from the bottom. In the most common technique, all of these trapezoids are then superimposed one upon another, forming a single geometric shape. Then, the centroid of this shape, called the fuzzy centroid, is calculated. The x coordinate of the centroid is the defuzzified value. == Methods == There are many different methods of defuzzification available, including the following: AI (adaptive integration) BADD (basic defuzzification distributions) BOA (bisector of area) CDD (constraint decision defuzzification) COA (center of area) COG (center of gravity) ECOA (extended center of area) EQM (extended quality method) FCD (fuzzy clustering defuzzification) FM (fuzzy mean) FOM (first of maximum) GLSD (generalized level set defuzzification) ICOG (indexed center of gravity) IV (influence value) LOM (last of maximum) MeOM (mean of maxima) MOM (middle of maximum) QM (quality method) RCOM (random choice of maximum) SLIDE (semi-linear defuzzification) WFM (weighted fuzzy mean) The maxima methods are good candidates for fuzzy reasoning systems. The distribution methods and the area methods exhibit the property of continuity that makes them suitable for fuzzy controllers.
Coupled pattern learner
Coupled Pattern Learner (CPL) is a machine learning algorithm which couples the semi-supervised learning of categories and relations to forestall the problem of semantic drift associated with boot-strap learning methods. == Coupled Pattern Learner == Semi-supervised learning approaches using a small number of labeled examples with many unlabeled examples are usually unreliable as they produce an internally consistent, but incorrect set of extractions. CPL solves this problem by simultaneously learning classifiers for many different categories and relations in the presence of an ontology defining constraints that couple the training of these classifiers. It was introduced by Andrew Carlson, Justin Betteridge, Estevam R. Hruschka Jr. and Tom M. Mitchell in 2009. == CPL overview == CPL is an approach to semi-supervised learning that yields more accurate results by coupling the training of many information extractors. Basic idea behind CPL is that semi-supervised training of a single type of extractor such as ‘coach’ is much more difficult than simultaneously training many extractors that cover a variety of inter-related entity and relation types. Using prior knowledge about the relationships between these different entities and relations CPL makes unlabeled data as a useful constraint during training. For e.g., ‘coach(x)’ implies ‘person(x)’ and ‘not sport(x)’. == CPL description == === Coupling of predicates === CPL primarily relies on the notion of coupling the learning of multiple functions so as to constrain the semi-supervised learning problem. CPL constrains the learned function in two ways. Sharing among same-arity predicates according to logical relations Relation argument type-checking === Sharing among same-arity predicates === Each predicate P in the ontology has a list of other same-arity predicates with which P is mutually exclusive. If A is mutually exclusive with predicate B, A’s positive instances and patterns become negative instances and negative patterns for B. For example, if ‘city’, having an instance ‘Boston’ and a pattern ‘mayor of arg1’, is mutually exclusive with ‘scientist’, then ‘Boston’ and ‘mayor of arg1’ will become a negative instance and a negative pattern respectively for ‘scientist.’ Further, Some categories are declared to be a subset of another category. For e.g., ‘athlete’ is a subset of ‘person’. === Relation argument type-checking === This is a type checking information used to couple the learning of relations and categories. For example, the arguments of the ‘ceoOf’ relation are declared to be of the categories ‘person’ and ‘company’. CPL does not promote a pair of noun phrases as an instance of a relation unless the two noun phrases are classified as belonging to the correct argument types. === Algorithm description === Following is a quick summary of the CPL algorithm. Input: An ontology O, and a text corpus C Output: Trusted instances/patterns for each predicate for i=1,2,...,∞ do foreach predicate p in O do EXTRACT candidate instances/contextual patterns using recently promoted patterns/instances; FILTER candidates that violate coupling; RANK candidate instances/patterns; PROMOTE top candidates; end end ==== Inputs ==== A large corpus of Part-Of-Speech tagged sentences and an initial ontology with predefined categories, relations, mutually exclusive relationships between same-arity predicates, subset relationships between some categories, seed instances for all predicates, and seed patterns for the categories. ==== Candidate extraction ==== CPL finds new candidate instances by using newly promoted patterns to extract the noun phrases that co-occur with those patterns in the text corpus. CPL extracts, Category Instances Category Patterns Relation Instances Relation Patterns ==== Candidate filtering ==== Candidate instances and patterns are filtered to maintain high precision, and to avoid extremely specific patterns. An instance is only considered for assessment if it co-occurs with at least two promoted patterns in the text corpus, and if its co-occurrence count with all promoted patterns is at least three times greater than its co-occurrence count with negative patterns. ==== Candidate ranking ==== CPL ranks candidate instances using the number of promoted patterns that they co-occur with so that candidates that occur with more patterns are ranked higher. Patterns are ranked using an estimate of the precision of each pattern. ==== Candidate promotion ==== CPL ranks the candidates according to their assessment scores and promotes at most 100 instances and 5 patterns for each predicate. Instances and patterns are only promoted if they co-occur with at least two promoted patterns or instances, respectively. == Meta-Bootstrap Learner == Meta-Bootstrap Learner (MBL) was also proposed by the authors of CPL. Meta-Bootstrap learner couples the training of multiple extraction techniques with a multi-view constraint, which requires the extractors to agree. It makes addition of coupling constraints on top of existing extraction algorithms, while treating them as black boxes, feasible. MBL assumes that the errors made by different extraction techniques are independent. Following is a quick summary of MBL. Input: An ontology O, a set of extractors ε Output: Trusted instances for each predicate for i=1,2,...,∞ do foreach predicate p in O do foreach extractor e in ε do Extract new candidates for p using e with recently promoted instances; end FILTER candidates that violate mutual-exclusion or type-checking constraints; PROMOTE candidates that were extracted by all extractors; end end Subordinate algorithms used with MBL do not promote any instance on their own, they report the evidence about each candidate to MBL and MBL is responsible for promoting instances. == Applications == In their paper authors have presented results showing the potential of CPL to contribute new facts to existing repository of semantic knowledge, Freebase
Key & See
Key & See is a variation of the TV Key service that forms part of the open, standards-based interactive TV services platform provided by Miniweb Interactive. Key & See allows viewers to access the interactive TV content made available by broadcasters and channel owners while leaving quarter of their screen tuned to the programme they are already watching Like TV Key, Key & See can be used with interactive TV services on UK satellite TV provider Sky Digital (BSkyB) Key & See works in the same way as a TV Key but the numeric shortcut code is associated with a broadcaster and a particular TV channel or programme. Miniweb Interactive offers commercial brands and broadcasters the chance to utilise TV Key and Key & See technology as part of its interactive TV services platform
Anonymous social media
Anonymous social media is a subcategory of social media wherein the main social function is to share and interact around content and information anonymously on mobile and web-based platforms. Another key aspect of anonymous social media is that content or information posted is not connected with particular online identities or profiles. == Background == Appearing very early on the web as mostly anonymous-confession websites, this genre of social media has evolved into various types and formats of anonymous self-expression. One of the earliest anonymous social media forums was 2channel, which was first introduced online on May 30, 1999, as a Japanese text board forum. With the way digital content is consumed and created continuously changing, the trending shift from web to mobile applications is also affecting anonymous social media. This can be seen as anonymous blogging, or various other format based content platforms such as nameless question and answer online platforms like Ask.fm introduced mobile versions of their services. The number of new networks joining the anonymous social sharing scene continues to grow rapidly. == Degrees of anonymity == Across different forms of anonymous social media there are varying degrees of anonymity. Some applications, such as Librex, require users to sign up for an account, even though their profile is not linked to their posts. While these applications remain anonymous, some of these sites can sync up with the user's contact list or location to develop a context within the social community and help personalize the user's experience, such as Yik Yak or Secret. Other sites, such as 4chan and 2channel, allow for a purer form of anonymity as users are not required to create an account, and posts default to the username of "Anonymous". While users can still be traced through their IP address, there are anonymizing services like I2P or various proxy server services that encrypt a user's identity online by running it through different routers. Secret users must provide a phone number or email when signing up for the service, and their information is encrypted into their posts. Stylometry poses a risk to the anonymity or pseudonymity of social media users, who may be identifiable by writing style; in turn, they may use adversarial stylometry to resist such identification. == Controversy == Apps such as Formspring, Ask, Sarahah, Whisper, and Secret have elicited discussion around the rising popularity of anonymity apps, including debate and anticipation about this social sharing class. As more and more platforms join the league of anonymous social media, there is growing concern about the ethics and morals of anonymous social networking as cases of cyber-bullying, and personal defamation occurs. Formspring, also known as spring.me, and Ask.fm have both been associated with teen suicides as a result of cyberbullying on the sites. Formspring has been associated with at least three teen suicides and Ask.fm with at least five. For instance, the app Secret got shut down due to its escalated use of cyberbullying. The app Yik Yak has also helped to contribute to more cyberbullying situations and, in turn, was blocked on some school networks. Their privacy policy meant that users could not be identified without a subpoena, search warrant, or court order. Another app called After School also sparked controversy for its app design that lets students post any anonymous content. Due to these multiple controversies, the app has been removed from both Apple and Google app stores. As the number of people using these platforms multiplies, unintended uses of the apps have increased, urging popular networks to enact in-app warnings and prohibit the use for middle and high school students. 70% of teens admit to making an effort to conceal their online behavior from their parents. Even Snapchat has some relation to the health of children after using social media. This is an app that is meant to be quick and simple but in many ways it can be overwhelming. A person can post something, and it will be gone in seconds. Oftentimes, the post that was made was inappropriate and harmful to another person. It's a never-ending cycle. Some of these apps have also been criticized for causing chaos in American schools, such as lockdowns and evacuations. In order to limit the havoc caused, anonymous apps are currently removing all abusive and harmful posts. Apps such as Yik Yak, Secret, and Whisper are removing these posts by outsourcing the job of content supervision to oversea surveillance companies. These companies hire a team of individuals to inspect and remove any harmful or abusive posts. Furthermore, algorithms are also used to detect and remove any abusive posts the individuals may have missed. Another method used by the anonymous app named Cloaq to reduce the number of harmful and abusive posts is to limit the number of users that can register during a certain period. Under this system, all contents are still available to the public, but only registered users can post. Other websites such as YouTube have gone on to create new policies regarding anonymity. YouTube now does not allow anonymous comments on videos. Users must have a Google account to like, dislike, comment or reply to comments on videos. Once a sign-in user "likes" a video, it will be added to that user's 'Liked video playlist'. YouTube changed their "Liked video playlist" policy in December 2019, allowing a signed-in user to keep their "Liked video playlist" private. Historically, these controversies and the rise of cyberbullying have been blamed on the anonymous aspect of many social media platforms, but about half of US adult online harassment cases do not involve anonymity, and researchers have found that if targeted harassment exists offline it will also be found online, because online harassment is a reflection of existing prejudices. == As platforms for anonymous discussion == Anonymous social media can be used for political discussion in countries where political opinions opposed to the government are normally suppressed, and allow persons of different genders to communicate freely in cultures where such communication is not generally accepted. In the United States, the 2016 presidential election led to an increase in the use of anonymous social media websites to express political stances. Moreover, anonymous social media can also provide authentic connection to complete anonymous communication. There have been cases where these anonymous platforms have saved individuals from life-threatening situation or spread news about a social cause. Additionally, anonymous social websites also allow internet users to communicate while also safeguarding personal information from criminal actors and corporations that sell users' data. A study in 2017 on the content posted to 4chan's /pol/ board found that the majority of the content was unique, including 70% of the 1 million images included in the studied data set. == Revenue generated by anonymous social media == === Anonymous apps === Generating revenue from anonymous apps has been a discussion for investors. Since little information is collected about the users, it is difficult for anonymous apps to advertise to users. However some apps, such as Whisper, have found a method to overcome this obstacle. They have developed a "keyword-based" approach, where advertisements are shown to users depending on certain words they type. The app Yik Yak has been able to capitalize on the features they provide. Anonymous apps such a Chrends take the approach of using anonymity to provide freedom of speech. Telephony app Burner has regularly been a top grossing utilities app in the iOS and Android app stores using its phone number generation technology. Despite the success of some anonymous apps, there are also apps, such as Secret, which have yet to find a way to generate revenue. The idea of an anonymous app has also caused mixed opinions within investors. Some investors have invested a large sum of money because they see the potential revenue generated within these apps. Other investors have stayed away from investing these apps because they feel these apps bring more harm than good. === Anonymous sites === There are several sources to generate revenue for anonymous social media sites. One source of revenue is by implementing programs such as a premium membership or a gift-exchanging program. Another source of revenue is by merchandising goods and specific usernames to users. In addition, sites such as FMyLife, have implemented a policy where the anonymous site will receive 50% of profit from apps that makes money off it. In terms of advertisements, some anonymous sites have had troubles implementing or attracting them. There are several reasons for this problem. Anonymous sites, such as 4chan, have received few advertisement offers due to some of the contents it generates. Other anonymous sites, such as Reddit, have been ca
Optical recording
The history of optical recording can be divided into a few number of distinct major contributions. The pioneers of optical recording worked mostly independently, and their solutions to the many technical challenges have very distinctive features, such as reflective disc (Compaan and Kramer) transparent disc (Gregg) floppy disc (Russell) rigid disc (Compaan and Kramer) focused laser beam for read-out through transparent substrate (Compaan and Kramer). == Gregg 1958 == Laserdisc technology, using a transparent disc, was invented by David Paul Gregg in 1958 (and patented in 1970 and 1990). By 1969 Philips had developed a videodisc in reflective mode, which has great advantages over the transparent mode. MCA and Philips decided to join their efforts. They first publicly demonstrated the videodisc in 1972. Laserdisc was first available on the market, in Atlanta, on December 15, 1978, two years after the VHS VCR and four years before the CD, which is based on Laserdisc technology. Philips produced the players and MCA produced the discs. The Philips/MCA cooperation was not successful, and discontinued after a few years. Several of the scientists responsible for the early research (John Winslow, Richard Wilkinson and Ray Dakin) founded Optical Disc Corporation (now ODC Nimbus). == Russell 1965 == While working at Pacific Northwest National Laboratory, James Russell invented an optical storage system for digital audio and video, patenting the concept in 1970. The earliest patents by Russell, US 3,501,586, and 3,795,902 were filed in 1966, and 1969. respectively. He built prototypes, and the first was operating in 1973. Russell had found a way to record digital information onto a photosensitive plate in tiny dark spots, each spot one micrometre from centre to centre, with a laser that wrote the binary patterns. Russell's first optical disc was distinctly different from the eventual compact disc product: the disc in the player was not read by laser light. A key characteristic of Russell's invention is that a laser is not used for the reading the disc, instead the entire disc or oblong sheet to be read is illuminated by a large playback light source at the back of the transparent foil. As a result, the information density is relatively low. By 1985, Russell held over 25 patents to various technologies related to optical recording and playback. Russell's intellectual property was purchased by Optical Recording Corporation (ORC) in Toronto in 1985, and this firm notified a number of CD manufacturers that their CD technology was based on patents held by ORC. In 1987, ORC signed an agreement with Sony whereby Sony paid for licensing of the technology. Further licenses followed from Philips and others. Warner Communications did not sign, and was sued by ORC. In 1992, the large CD manufacturer, now called Time Warner, was ordered to pay ORC US$30 million in patent violations. In the 1970 patent, the spot diameter was around 10 micrometres. Thus, the areal information density was around a factor hundred less than that of the CD as later developed. Russell continued to refine the concept throughout the 1970s. Philips and Sony, however, were able to put far greater resources into the parallel development of the concept, arriving at a smaller and more sophisticated product in just a few years. Russell's various partners and ventures failed to produce a single consumer product. == Korpel 1968 == Adrianus Korpel worked for the Zenith Electronics Corporation, when he developed very early optical videodisc systems, including holographic storage. == Kramer and Compaan 1969 == The Philips development of the videodisc technology began in 1969 with efforts by Dutch physicists Klaas Compaan and Piet Kramer to record video images in holographic form on disc. Their prototype Laserdisc shown in 1972 used a laser beam in reflective mode to read a track of pits using an FM video signal. Together with MCA, Philips brought the optical videodisk to market in 1978. The cooperation between Philips and MCA did not last long, and discontinued after a few years. == Immink and Doi 1979 == The Compact Disc (CD), which is based on MCA/Philips Laserdisc technology, was developed by a taskforce of Sony and Philips in 1979–1980. Toshi Doi and Kees Schouhamer Immink created the digital technologies that turned the analog Laserdisc into a high-density low-cost digital audio disc. The CD, available on the market since October 1982, remains the standard physical medium for sale of commercial audio recordings Standard CDs have a diameter of 120 mm and can hold up to 80 minutes of audio (700 MB of data). The Mini CD has various diameters ranging from 60 to 80 mm; they are sometimes used for CD singles or device drivers, storing up to 24 minutes of audio. The technology was later adapted and expanded to include data storage CD-ROM, write-once audio and data storage CD-R, rewritable media CD-RW, Super Audio CD (SACD), Video Compact Discs (VCD), Super Video Compact Discs (SVCD), PhotoCD, PictureCD, CD-i, and Enhanced CD. CD-ROMs and CD-Rs remain widely used technologies in the computer industry. The CD and its extensions have been extremely successful: in 2004, worldwide sales of CD audio, CD-ROM, and CD-R reached about 30 billion discs. By 2007, 200 billion CDs had been sold worldwide.
Attention (machine learning)
In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In natural language processing, importance is represented by "soft" weights assigned to each word in a sentence. More generally, attention encodes vectors called token embeddings across a fixed-width sequence that can range from tens to millions of tokens in size. Unlike "hard" weights, which are computed during the backwards training pass, "soft" weights exist only in the forward pass and therefore change with every step of the input. Earlier designs implemented the attention mechanism in a serial recurrent neural network (RNN) language translation system, but a more recent design, namely the transformer, removed the slower sequential RNN and relied more heavily on the faster parallel attention scheme. Inspired by ideas about attention in humans, the attention mechanism was developed to address the weaknesses of using information from the hidden layers of recurrent neural networks. Recurrent neural networks favor information contained in words at the end of a sentence and thus deemed more recent, thereby tending to attenuate the significance and associated predictive weight assigned to information earlier in the sentence. Attention allows a token equal access to any part of a sentence directly, rather than only through the previous state. == History == Additional surveys of the attention mechanism in deep learning are provided by Niu et al. and Soydaner. The major breakthrough came with self-attention, where each element in the input sequence attends to all others, enabling the model to capture global dependencies. This idea was central to the Transformer architecture, which replaced recurrence with attention mechanisms. As a result, Transformers became the foundation for models like BERT, T5 and generative pre-trained transformers (GPT). == Overview == The modern era of machine attention was revitalized by grafting an attention mechanism (Fig 1. orange) to an Encoder-Decoder. Figure 2 shows the internal step-by-step operation of the attention block (A) in Fig 1. === Interpreting attention weights === In translating between languages, alignment is the process of matching words from the source sentence to words of the translated sentence. Networks that perform verbatim translation without regard to word order would show the highest scores along the (dominant) diagonal of the matrix. The off-diagonal dominance shows that the attention mechanism is more nuanced. Consider an example of translating I love you to French. On the first pass through the decoder, 94% of the attention weight is on the first English word I, so the network offers the word je. On the second pass of the decoder, 88% of the attention weight is on the third English word you, so it offers t'. On the last pass, 95% of the attention weight is on the second English word love, so it offers aime. In the I love you example, the second word love is aligned with the third word aime. Stacking soft row vectors together for je, t', and aime yields an alignment matrix: Sometimes, alignment can be multiple-to-multiple. For example, the English phrase look it up corresponds to cherchez-le. Thus, "soft" attention weights work better than "hard" attention weights (setting one attention weight to 1, and the others to 0), as we would like the model to make a context vector consisting of a weighted sum of the hidden vectors, rather than "the best one", as there may not be a best hidden vector. == Variants == Many variants of attention implement soft weights, such as fast weight programmers, or fast weight controllers (1992). A "slow" neural network outputs the "fast" weights of another neural network through outer products. The slow network learns by gradient descent. It was later renamed as "linearized self-attention". Bahdanau-style attention, also referred to as additive attention, Luong-style attention, which is known as multiplicative attention, Early attention mechanisms similar to modern self-attention were proposed using recurrent neural networks. However, the highly parallelizable self-attention was introduced in 2017 and successfully used in the Transformer model, positional attention and factorized positional attention. For convolutional neural networks, attention mechanisms can be distinguished by the dimension on which they operate, namely: spatial attention, channel attention, or combinations. These variants recombine the encoder-side inputs to redistribute those effects to each target output. Often, a correlation-style matrix of dot products provides the re-weighting coefficients. In the figures below, W is the matrix of context attention weights, similar to the formula in Overview section above. == Optimizations == === Flash attention === The size of the attention matrix is proportional to the square of the number of input tokens. Therefore, when the input is long, calculating the attention matrix requires a lot of GPU memory. Flash attention is an implementation that reduces the memory needs and increases efficiency without sacrificing accuracy. It achieves this by partitioning the attention computation into smaller blocks that fit into the GPU's faster on-chip memory, reducing the need to store large intermediate matrices and thus lowering memory usage while increasing computational efficiency. === FlexAttention === FlexAttention is an attention kernel developed by Meta that allows users to modify attention scores prior to softmax and dynamically chooses the optimal attention algorithm. == Applications == Attention is widely used in natural language processing, computer vision, and speech recognition. In NLP, it improves context understanding in tasks like question answering and summarization. In vision, visual attention helps models focus on relevant image regions, enhancing object detection and image captioning. === Attention maps as explanations for vision transformers === From the original paper on vision transformers (ViT), visualizing attention scores as a heat map (called saliency maps or attention maps) has become an important and routine way to inspect the decision making process of ViT models. One can compute the attention maps with respect to any attention head at any layer, while the deeper layers tend to show more semantically meaningful visualization. Attention rollout is a recursive algorithm to combine attention scores across all layers, by computing the dot product of successive attention maps. Because vision transformers are typically trained in a self-supervised manner, attention maps are generally not class-sensitive. When a classification head is attached to the ViT backbone, class-discriminative attention maps (CDAM) combines attention maps and gradients with respect to the class [CLS] token. Some class-sensitive interpretability methods originally developed for convolutional neural networks can be also applied to ViT, such as GradCAM, which back-propagates the gradients to the outputs of the final attention layer. Using attention as basis of explanation for the transformers in language and vision is not without debate. While some pioneering papers analyzed and framed attention scores as explanations, higher attention scores do not always correlate with greater impact on model performances. == Mathematical representation == === Standard scaled dot-product attention === For matrices: Q ∈ R m × d k , K ∈ R n × d k {\displaystyle Q\in \mathbb {R} ^{m\times d_{k}},K\in \mathbb {R} ^{n\times d_{k}}} and V ∈ R n × d v {\displaystyle V\in \mathbb {R} ^{n\times d_{v}}} , the scaled dot-product, or QKV attention, is defined as: Attention ( Q , K , V ) = softmax ( Q K T d k ) V ∈ R m × d v {\displaystyle {\text{Attention}}(Q,K,V)={\text{softmax}}\left({\frac {QK^{T}}{\sqrt {d_{k}}}}\right)V\in \mathbb {R} ^{m\times d_{v}}} where T {\displaystyle {}^{T}} denotes transpose and the softmax function is applied independently to every row of its argument. The matrix Q {\displaystyle Q} contains m {\displaystyle m} queries, while matrices K , V {\displaystyle K,V} jointly contain an unordered set of n {\displaystyle n} key-value pairs. Value vectors in matrix V {\displaystyle V} are weighted using the weights resulting from the softmax operation, so that the rows of the m {\displaystyle m} -by- d v {\displaystyle d_{v}} output matrix are confined to the convex hull of the points in R d v {\displaystyle \mathbb {R} ^{d_{v}}} given by the rows of V {\displaystyle V} . To understand the permutation invariance and permutation equivariance properties of QKV attention, let A ∈ R m × m {\displaystyle A\in \mathbb {R} ^{m\times m}} and B ∈ R n × n {\displaystyle B\in \mathbb {R} ^{n\times n}} be permutation matrices; and D ∈ R m × n {\displaystyle D\in \mathbb {R} ^{m\times n}} an arbitrary matrix. The softmax function is permutation equivariant in the sense that: softmax ( A D B ) = A softmax ( D ) B {\displays
Webedia
Webedia S.A. is a company specializing in online media, a subsidiary of the Fimalac group based in Levallois-Perret, France. Webedia is active in more than twenty countries including France (AlloCiné, Jeuxvideo.com, MGG, Puremédias, Ode, Pureshopping, Volum, Terrafemina, 750g, easyVoyage, l’Automobile Magazine, Le 10 Sport), Brazil (AdoroCinema, Tudo Gostoso, Minhavida), Germany (Filmstarts, Moviepilot, GameStar), Spain and Latin America (Xataka, SensaCine, Raiser Games), Poland (Gry-Online and GetHero) and the United States (Boxoffice Pro). == History == === Early years (2007-2013) === Webedia was created in France in 2007, following the successive launches of the websites Purepeople, Puretrend and Purefans. Webedia bought the comparison shopping website Shopoon in 2008 and renamed it Pureshopping, and the website Ozap (media news) from M6 group in 2011 and renamed it Puremédias. Webedia was acquired by Fimalac in May 2013 and became its Internet media subsidiary. === Growth (2013-2016) === In 2013, Fimalac acquired AlloCiné, the websites Newsring and Youmag, the cooking website 750g and the cultural platform Exponaute. In 2014, Webedia acquired OverBlog, Jeuxvideo.com (through L'Odyssée Interactive and moved to Paris in 2015), Moviepilot (Germany), and Gameo Consulting (owner of Millenium, electronic sports), In December 2014, Webedia announced a license agreement with Ziff Davis to launch sites under the IGN franchise in Brazil and France at the beginning of 2015. The French version of IGN was launched on 2, it targets the general public and casual gamers. In 2015, Webedia acquired Côté Ciné Group (technological solutions for movie theaters and specialized press magazines: BoxOffice Pro in the United States and Côté Ciné in France), 57% of Easyvoyage group (online travel comparators Easyvol and Alibabuy, Mixicom (website JeuxActu and multi-channel network), 50% of the Brazilian network Paramaker, and West World Media (digital marketing company for the film industry). In 2016, Webedia bought Scimob (mobile video game studio), Surprizemi (home-delivered surprise boxes), Eklablog (blogging platform) Oxent (eSports World Convention), and Bang Bang Management (sports PR agency). In addition, an agreement is made with Paris Saint-Germain for Webedia to recruit and manage e-sports players on behalf of Paris Saint-Germain eSports. On November 15, 2016, the LFP announced that it had reached an agreement with beIN Sports and Webedia for the broadcasting of the first edition of the e-League 1. The competition is renewed for two additional seasons on July 26, 2017, the broadcasting agreements are renewed. On December 8, 2016, Webedia joined forces with Chronopost to launch Pourdebon, a home delivery service that connects Internet users and labeled producers (AOC, organic AB, etc.). Webedia has a slight majority (53%) in this new platform. === 2017 === On January 19, 2017, Webedia announced the acquisition of the English company Peach Digital, specializing in web development and digital marketing for movie theaters. In February 2017, Le Figaro announced that Webedia had invested 10 million euros in Illico Fresco, a home delivery service for baskets of recipes. The same month, FDJ and Webedia announced a partnership for the creation of eSports competitions: a professional one (FDJ Masters League) and another one for amateur gamers (FDJ Open Series) starting in March 2017. They are broadcast on Webedia's Web TV. At the end of February 2017, the media group finalized the acquisition of MyPoseo, a SaaS publisher specialized on SEO analytics. On March 8, 2017, Webedia launched LeStream, a Twitch Web TV dedicated to video games, the result of two years of development, in the company of several YouTubers including Cyprien and Squeezie,. On March 29, 2017, Webedia bought the Brazilian web publisher Minha Vida, a website devoted to health, nutrition, beauty and fitness, which attracts 14.3 million unique monthly visitors. Webedia reaches 44 million unique visitors in Brazil, and thus becomes the leading publisher on entertainment themes. In June 2017, the company made its largest international acquisition, with the American agency 3BlackDot, a media and marketing agency focused on videogamers. The agency, based in Los Angeles, manages 36 YouTubers followed by millions of subscribers on their channels which total 700 million videos viewed per month. In July 2017, Webedia bought IDZ, an audiovisual production company, and thus strengthened its production activities and its leadership on the YouTube channel networks in France. That year, Webedia was the first French media group to use the measurement of their global audiences by Comscore. It represents deduplicated coverage on desktops, laptops, smartphones and tablets, and includes audiences for websites, mobile applications and videos. This new measure allows Webedia to establish a deduplicated global audience of 177 million unique visitors in April 2017. In October 2017, Webedia announced its intention to launch a TV channel dedicated to electronic sports, called ES1. The channel was officially launched on January 10, 2018, on Orange TV and on February 6, 2018, on Free and Bouygues Telecom. In November 2017, Webedia, with the support of CDC International Capital, entered into exclusive negotiations with the Saudi company Uturn Entertainment, specializing in online entertainment, particularly on YouTube, and the production of digital content for the region's youth, with a view to merging it with Diwanee, a Webedia subsidiary in the Middle East, for an amount close to $100 million. In December 2017, Webedia acquired a majority stake in the United States–based company called Creators Media, which brings together social and video production platforms specializing in popular culture and entertainment. That same month, Webedia joined forces with Elephant, Emmanuel Chain's audiovisual production company, to create a new content production label aimed at Millennials. === 2018-2019 === In January 2018, Webedia launched a sports marketing agency: Only Sports & Passions. That same month, Illico Fresco, specialist in the delivery of kit meals belonging to Webedia, joined forces with Weight Watchers, the world leader in slimming products. In April 2018, Webedia published new audience figures in partnership with Comscore, 188 million unique monthly visitors in December 2017, an increase of 6.2% compared to the previous measure dating from April 2017. The same month, Webedia unveils its ambitions concerning content production, as a partnership with the video game studio Focus Home Interactive is signed with a title "Fear the Wolves" already planned for 2018, co-production projects of films, cartoons or series are announced. In July 2018, Webedia bought the American authors company Full Fathom Five, a company that helps authors produce books, TV series, films and video games. In October 2018, Webedia announced that it was focusing on both esports clubs PSG Esports and LeStream Esport. The first one being geared towards international competitions and the second devoted mainly to the French esports scene. The "Millenium" brand is thus refocusing around its media activities and esports merchandising products, and the "Millenium esport club" being gradually closed. The same month, the company announced the acquisition of Weblogs, a Spanish-speaking website publisher, thereby strengthening its activity in Spain and Latin America. On October 22, 2018, Webedia announced the merger of BoxOffice magazine with Film Journal International. On November 13, 2018, Groupe SEB announced the acquisition from Webedia of 750g International, the international branch of the French recipe site 750g (the original French website 750g.com being retained by Webedia). The group is thus separating from Gourmandize (United States and United Kingdom), HeimGourmet (Germany), Rebañando (Spain), Receitas Sem Fronteiras (Brazil / Portugal) and Tribù Golosa (Italy). The same month, Webedia joined forces with Riot Games to launch the French League of League of Legends (LFL), the first French professional league on the League of Legends game, which will bring together the 8 best teams on the French scene. In March 2019, Webedia bought 51% of the audiovisual production company Elephant. The new set will weigh 500 million euros, a quarter of which will be made outside France. The same month, Webedia purchased a majority stake in the company Partoo, which publishes a SaaS platform specializing in local marketing for brands and merchants. On March 14, 2019, a new measurement of the international audience of Webedia sites was produced by Comscore, posting 250 million unique visitors in December 2018, up 9.2% compared to December 2017. In June 2019, the group joined forces with Michel Cymes, a famous doctor and French TV host by taking a majority stake in his company Club Santé Débat, in order to develop a health platform around the Dr. Good! Brand. In Sep