Expectation propagation (EP) is a technique in Bayesian machine learning. EP finds approximations to a probability distribution. It uses an iterative approach that uses the factorization structure of the target distribution. It differs from other Bayesian approximation approaches such as variational Bayesian methods. More specifically, suppose we wish to approximate an intractable probability distribution p ( x ) {\displaystyle p(\mathbf {x} )} with a tractable distribution q ( x ) {\displaystyle q(\mathbf {x} )} . Expectation propagation achieves this approximation by minimizing the Kullback–Leibler divergence K L ( p | | q ) {\displaystyle \mathrm {KL} (p||q)} . Variational Bayesian methods minimize K L ( q | | p ) {\displaystyle \mathrm {KL} (q||p)} instead. If q ( x ) {\displaystyle q(\mathbf {x} )} is a Gaussian N ( x | μ , Σ ) {\displaystyle {\mathcal {N}}(\mathbf {x} |\mu ,\Sigma )} , then K L ( p | | q ) {\displaystyle \mathrm {KL} (p||q)} is minimized with μ {\displaystyle \mu } and Σ {\displaystyle \Sigma } being equal to the mean of p ( x ) {\displaystyle p(\mathbf {x} )} and the covariance of p ( x ) {\displaystyle p(\mathbf {x} )} , respectively; this is called moment matching. == Applications == Expectation propagation via moment matching plays a vital role in approximation for indicator functions that appear when deriving the message passing equations for TrueSkill.
Web development tools
Web development tools (often abbreviated to dev tools) allow web developers to test, modify and debug their websites. They are different from website builders and integrated development environments (IDEs) in that they do not assist in the direct creation of a webpage, rather they are tools used for testing the user interface of a website or web application. Web development tools come as browser add-ons or built-in features in modern web browsers. Browsers such as Google Chrome, Firefox, Safari, Microsoft Edge, and Opera have built-in tools to help web developers, and many additional add-ons can be found in their respective plugin download centers. Web development tools allow developers to work with a variety of web technologies, including HTML, CSS, the DOM, JavaScript, and other components that are handled by the web browser. == History and support == Early web developers manually debugged their websites by commenting out code and using JavaScript functions. One of the first browser debugging tools to exist was Mozilla's Firebug extension, which possessed many of the current core features of today's developer tools, leading to Firefox becoming popular with developers at the time. Safari's WebKit engine also introduced its integrated developer tools around that period, which eventually became the basis for both Safari and Chrome's current tooling. Microsoft released a developer toolbar for Internet Explorer 6 and 7; and then integrated them into the browser from version 8 onwards. In 2017, Mozilla discontinued Firebug in favour of integrated developer tools. Nowadays, all modern web browsers have support for web developer tools that allow web designers and developers to look at the make-up of their pages. These are all tools that are built into the browser and do not require additional modules or configuration. Firefox – F12 opens the Firefox DevTools. Google Chrome and Opera – Developer Tools (DevTools) Microsoft Edge – F12 opens Web Developer Tools. Microsoft incorporates additional features that are not included in mainline Chromium. Safari – The Safari Web Inspector has to be enabled from its settings pane. == Features == The built-in web developer tools in the browser are commonly accessed by hovering over an item on a webpage and selecting the "Inspect Element" or similar option from the context menu. Alternatively the F12 key tends to be another common shortcut. === HTML and the DOM === HTML and DOM viewer and editor is commonly included in the built-in web development tools. The difference between the HTML and DOM viewer, and the view source feature in web browsers is that the HTML and DOM viewer allows you to see the DOM as it was rendered in addition to allowing you to make changes to the HTML and DOM and see the change reflected in the page after the change is made. In addition to selecting and editing, the HTML elements panels will usually also display properties of the DOM object, such as display dimension, and CSS properties. Firefox, Safari, Chrome, and Edge all allow users to simulate the document on a mobile device by modifying the viewport dimensions and pixel density. Additionally, Firefox and Chrome both have the option to simulate colour blindness for the page. === Web page assets, resources and network information === Web pages typically load and require additional content in the form of images, scripts, font and other external files. Web development tools also allow developers to inspect resources that are loaded and available on the web page in a tree-structure listing, and the appearance of style sheets can be tested in real time. Web development tools also allow developers to view information about the network usage, such as viewing what the loading time and bandwidth usage are and which HTTP headers are being sent and received. Developers can manipulate and resend network requests. === Profiling and auditing === Profiling allows developers to capture information about the performance of a web page or web application. With this information developers can improve the performance of their scripts. Auditing features may provide developers suggestions, after analyzing a page, for optimizations to decrease page load time and increase responsiveness. Web development tools typically also provide a record of the time it takes to render the page, memory usage, and the types of events which are taking place. These features allow developers to optimize their web page or web application. ==== JavaScript debugging ==== JavaScript is commonly used in web browsers. Web development tools commonly include a debugger panel for scripts by allowing developers to add watch expressions, breakpoints, view the call stack, and pause, continue, and step while debugging JavaScript. A console is also often included, which allow developers to type in JavaScript commands and call functions, or view errors that may have been encountered during the execution of a script. === Extensions === The devtools API allows browser extensions to add their own features to developer tools.
Laws of Form
Laws of Form (hereinafter LoF) is a book by G. Spencer-Brown, written by August 1967 and published in 1969. The book straddles the boundary between mathematics and philosophy. LoF describes three distinct logical systems: The primary arithmetic (described in Chapter 4 of LoF), whose models include Boolean arithmetic; The primary algebra (Chapter 6 of LoF), whose models include the two-element Boolean algebra (hereinafter abbreviated 2), Boolean logic, and the classical propositional calculus; Equations of the second degree (Chapter 11), whose interpretations include finite automata and Alonzo Church's Restricted Recursive Arithmetic (RRA). "Boundary algebra" is a Meguire (2011) term for the union of the primary algebra and the primary arithmetic. Laws of Form sometimes loosely refers to the "primary algebra" as well as to LoF. == Contents == The preface states that the work was first explored in 1959, and Spencer Brown cites Bertrand Russell as being supportive of his endeavour. He also thanks J. C. P. Miller of University College London for helping with the proofreading and offering other guidance. In 1963 Spencer Brown was invited by Harry Frost, staff lecturer in the physical sciences at the department of Extra-Mural Studies of the University of London, to deliver a course on the mathematics of logic. LoF emerged from work in electronic engineering its author did around 1960. Key ideas of the LOF were first outlined in his 1961 manuscript Design with the Nor, which remained unpublished until 2021, and further refined during subsequent lectures on mathematical logic he gave under the auspices of the University of London's extension program. LoF has appeared in several editions. The second series of editions appeared in 1972 with the "Preface to the First American Edition", which emphasised the use of self-referential paradoxes, and the most recent being a 1997 German translation. LoF has never gone out of print. LoF's mystical and declamatory prose and its love of paradox make it a challenging read for all. Spencer-Brown was influenced by Ludwig Wittgenstein and R. D. Laing. LoF also echoes a number of themes from the writings of Charles Sanders Peirce, Bertrand Russell, and Alfred North Whitehead. The work has had curious effects on some classes of its readership; for example, on obscure grounds, it has been claimed that the entire book is written in an operational way, giving instructions to the reader instead of telling them what "is", and that in accordance with G. Spencer-Brown's interest in paradoxes, the only sentence that makes a statement that something is, is the statement which says no such statements are used in this book. Furthermore, the claim asserts that except for this one sentence the book can be seen as an example of E-Prime. What prompted such a claim, is obscure, either in terms of incentive, logical merit, or as a matter of fact, because the book routinely and naturally uses the verb to be throughout, and in all its grammatical forms, as may be seen both in the original and in quotes shown below. == Reception == Ostensibly a work of formal mathematics and philosophy, LoF became something of a cult classic: it was praised by Heinz von Foerster when he reviewed it for the Whole Earth Catalog. Those who agree point to LoF as embodying an enigmatic "mathematics of consciousness", its algebraic symbolism capturing an (perhaps even "the") implicit root of cognition: the ability to "distinguish". LoF argues that primary algebra reveals striking connections among logic, Boolean algebra, and arithmetic, and the philosophy of language and mind. Stafford Beer wrote in a review for Nature in 1969, "When one thinks of all that Russell went through sixty years ago, to write the Principia, and all we his readers underwent in wrestling with those three vast volumes, it is almost sad". Banaschewski (1977) argues that the primary algebra is nothing but new notation for Boolean algebra. Indeed, the two-element Boolean algebra 2 can be seen as the intended interpretation of the primary algebra. Yet the notation of the primary algebra: Fully exploits the duality characterizing not just Boolean algebras but all lattices; Highlights how syntactically distinct statements in logic and 2 can have identical semantics; Dramatically simplifies Boolean algebra calculations, and proofs in sentential and syllogistic logic. Moreover, the syntax of the primary algebra can be extended to formal systems other than 2 and sentential logic, resulting in boundary mathematics. LoF has influenced, among others, Heinz von Foerster, Louis Kauffman, Niklas Luhmann, Humberto Maturana, Francisco Varela and William Bricken. Some of these authors have modified the primary algebra in a variety of interesting ways. LoF claimed that certain well-known mathematical conjectures of very long standing, such as the four color theorem, Fermat's Last Theorem, and the Goldbach conjecture, are provable using extensions of the primary algebra. Spencer-Brown eventually circulated a purported proof of the four color theorem, but it was met with skepticism. == The form (Chapter 1) == The symbol: Also called the "mark" or "cross", is the essential feature of the Laws of Form. In Spencer-Brown's inimitable and enigmatic fashion, the Mark symbolizes the root of cognition, i.e., the dualistic Mark indicates the capability of differentiating a "this" from "everything else but this". In LoF, a Cross denotes the drawing of a "distinction", and can be thought of as signifying the following, all at once: The act of drawing a boundary around something, thus separating it from everything else; That which becomes distinct from everything by drawing the boundary; Crossing from one side of the boundary to the other. All three ways imply an action on the part of the cognitive entity (e.g., person) making the distinction. As LoF puts it: "The first command: Draw a distinction can well be expressed in such ways as: Let there be a distinction, Find a distinction, See a distinction, Describe a distinction, Define a distinction, Or: Let a distinction be drawn". (LoF, Notes to chapter 2) The counterpoint to the Marked state is the Unmarked state, which is simply nothing, the void, or the un-expressable infinite represented by a blank space. It is simply the absence of a Cross. No distinction has been made and nothing has been crossed. The Marked state and the void are the two primitive values of the Laws of Form. The Cross can be seen as denoting the distinction between two states, one "considered as a symbol" and another not so considered. From this fact arises a curious resonance with some theories of consciousness and language. Paradoxically, the Form is at once Observer and Observed, and is also the creative act of making an observation. LoF (excluding back matter) closes with the words: ...the first distinction, the Mark and the observer are not only interchangeable, but, in the form, identical. C. S. Peirce came to a related insight in the 1890s; see § Related work. == The primary arithmetic (Chapter 4) == The syntax of the primary arithmetic goes as follows. There are just two atomic expressions: The empty Cross ; All or part of the blank page (the "void"). There are two inductive rules: A Cross may be written over any expression; Any two expressions may be concatenated. The semantics of the primary arithmetic are perhaps nothing more than the sole explicit definition in LoF: "Distinction is perfect continence". Let the "unmarked state" be a synonym for the void. Let an empty Cross denote the "marked state". To cross is to move from one value, the unmarked or marked state, to the other. We can now state the "arithmetical" axioms A1 and A2, which ground the primary arithmetic (and hence all of the Laws of Form): "A1. The law of Calling". Calling twice from a state is indistinguishable from calling once. To make a distinction twice has the same effect as making it once. For example, saying "Let there be light" and then saying "Let there be light" again, is the same as saying it once. Formally: = {\displaystyle \ =} "A2. The law of Crossing". After crossing from the unmarked to the marked state, crossing again ("recrossing") starting from the marked state returns one to the unmarked state. Hence recrossing annuls crossing. Formally: = {\displaystyle \ =} In both A1 and A2, the expression to the right of '=' has fewer symbols than the expression to the left of '='. This suggests that every primary arithmetic expression can, by repeated application of A1 and A2, be simplified to one of two states: the marked or the unmarked state. This is indeed the case, and the result is the expression's "simplification". The two fundamental metatheorems of the primary arithmetic state that: Every finite expression has a unique simplification. (T3 in LoF); Starting from an initial marked or unmarked state, "complicating" an expression by a finite number of repeated application of A1 and A2 cannot yield
Stefan Schaal
Stefan Schaal (born 1961) is a German-American computer scientist specializing in robotics, machine learning, autonomous systems, and computational neuroscience. == Education and career == Schaal was born in Frankfurt am Main in Germany, Schaal grew up in the North Bavarian town of Nürnberg. After graduating from school, he served in the German army in the Ski Patrol Division of Bad Reichenhall, where he honorably discharged with the rank of a Lieutenant. Schaal studied mechanical engineering at the Technical University of Munich, graduating in 1987 with a Diploma degree (summa cum laude). Subsequently, Schaal did his Ph.D. in computer aided design and artificial intelligence at the Technical University of Munich and the Massachusetts Institute of Technology, receiving his Ph.D. in 1991 (Summa Cum Laude) under Klaus Ehrlenspiel. In 1991, Schaal was a Postdoctoral Fellow at the Department and Brain and Cognitive Science and the Artificial Intelligence Lab at the Massachusetts Institute of Technology, funded by the Alexander von Humboldt Foundation and the German Academic Scholarship Foundation. Starting from 1992, he became an invited researcher at the ATR Computational Neuroscience Labs in Japan, where he created a robotics lab focusing on biological principles of motor control and learning. In 1994, Schaal moved to the Georgia Institute of Technology as an adjunct assistant professor, and also held the same rank at the Pennsylvania State University. In 1996, Schaal assumed a group leader position in the ERATO Kawato Dynamic Brain Project in Japan. Schaal joined the University of Southern California (USC) in 1997, where he advanced from the ranks of assistant professor, to associate professor, to full professor. In 2009, Schaal became a founder in defining and creating the Max Planck Institute for Intelligent Systems in Tübingen and Stuttgart, Germany, an institute focusing on principles of perception-action-learning systems in synthetic intelligence. In 2012, Schaal founded the Autonomous Motion Department (AMD) at this institute, while maintaining a partial appointment at USC. Stefan Schaal joined Google X as lead of a robotics research team in late 2018. == Research == Stefan Schaal's interests focus on autonomous perception-action-learning systems, in particular anthropomorphic robotic systems. He works on topics of machine learning for control, control theory, computational neuroscience for neuromotor control, experimental robotics, reinforcement learning, artificial intelligence, and nonlinear dynamical systems. Stefan has co-authored more than 400 publications in top conferences and journals, and served as organizer on various top conferences in machine learning and robotics. He has received numerous best paper awards and honors in his scientific community. Stefan Schaal has been noted as one of the five leaders in robotics in 2011, and among the top robotics experts in the world. == Controversy == In 2018, the German newsjournal Der Spiegel published an article reporting on his double affiliation with USC and the Max-Planck Society, both with full salaries, which was apparently unknown to either party. Schaal rejected the allegations, but was forced to leave his position at the Max Planck Institute.
Wasserstein GAN
The Wasserstein Generative Adversarial Network (WGAN) is a variant of generative adversarial network (GAN) proposed in 2017 that aims to "improve the stability of learning, get rid of problems like mode collapse, and provide meaningful learning curves useful for debugging and hyperparameter searches". Compared with the original GAN discriminator, the Wasserstein GAN discriminator provides a better learning signal to the generator. This allows the training to be more stable when generator is learning distributions in very high dimensional spaces. == Motivation == === The GAN game === The original GAN method is based on the GAN game, a zero-sum game with 2 players: generator and discriminator. The game is defined over a probability space ( Ω , B , μ r e f ) {\displaystyle (\Omega ,{\mathcal {B}},\mu _{ref})} , The generator's strategy set is the set of all probability measures μ G {\displaystyle \mu _{G}} on ( Ω , B ) {\displaystyle (\Omega ,{\mathcal {B}})} , and the discriminator's strategy set is the set of measurable functions D : Ω → [ 0 , 1 ] {\displaystyle D:\Omega \to [0,1]} . The objective of the game is L ( μ G , D ) := E x ∼ μ r e f [ ln D ( x ) ] + E x ∼ μ G [ ln ( 1 − D ( x ) ) ] . {\displaystyle L(\mu _{G},D):=\mathbb {E} _{x\sim \mu _{ref}}[\ln D(x)]+\mathbb {E} _{x\sim \mu _{G}}[\ln(1-D(x))].} The generator aims to minimize it, and the discriminator aims to maximize it. A basic theorem of the GAN game states that Repeat the GAN game many times, each time with the generator moving first, and the discriminator moving second. Each time the generator μ G {\displaystyle \mu _{G}} changes, the discriminator must adapt by approaching the ideal D ∗ ( x ) = d μ r e f d ( μ r e f + μ G ) . {\displaystyle D^{}(x)={\frac {d\mu _{ref}}{d(\mu _{ref}+\mu _{G})}}.} Since we are really interested in μ r e f {\displaystyle \mu _{ref}} , the discriminator function D {\displaystyle D} is by itself rather uninteresting. It merely keeps track of the likelihood ratio between the generator distribution and the reference distribution. At equilibrium, the discriminator is just outputting 1 2 {\displaystyle {\frac {1}{2}}} constantly, having given up trying to perceive any difference. Concretely, in the GAN game, let us fix a generator μ G {\displaystyle \mu _{G}} , and improve the discriminator step-by-step, with μ D , t {\displaystyle \mu _{D,t}} being the discriminator at step t {\displaystyle t} . Then we (ideally) have L ( μ G , μ D , 1 ) ≤ L ( μ G , μ D , 2 ) ≤ ⋯ ≤ max μ D L ( μ G , μ D ) = 2 D J S ( μ r e f ‖ μ G ) − 2 ln 2 , {\displaystyle L(\mu _{G},\mu _{D,1})\leq L(\mu _{G},\mu _{D,2})\leq \cdots \leq \max _{\mu _{D}}L(\mu _{G},\mu _{D})=2D_{JS}(\mu _{ref}\|\mu _{G})-2\ln 2,} so we see that the discriminator is actually lower-bounding D J S ( μ r e f ‖ μ G ) {\displaystyle D_{JS}(\mu _{ref}\|\mu _{G})} . === Wasserstein distance === Thus, we see that the point of the discriminator is mainly as a critic to provide feedback for the generator, about "how far it is from perfection", where "far" is defined as Jensen–Shannon divergence. Naturally, this brings the possibility of using a different criteria of farness. There are many possible divergences to choose from, such as the f-divergence family, which would give the f-GAN. The Wasserstein GAN is obtained by using the Wasserstein metric, which satisfies a "dual representation theorem" that renders it highly efficient to compute: A proof can be found in the main page on Wasserstein metric. == Definition == By the Kantorovich-Rubenstein duality, the definition of Wasserstein GAN is clear:A Wasserstein GAN game is defined by a probability space ( Ω , B , μ r e f ) {\displaystyle (\Omega ,{\mathcal {B}},\mu _{ref})} , where Ω {\displaystyle \Omega } is a metric space, and a constant K > 0 {\displaystyle K>0} . There are 2 players: generator and discriminator (also called "critic"). The generator's strategy set is the set of all probability measures μ G {\displaystyle \mu _{G}} on ( Ω , B ) {\displaystyle (\Omega ,{\mathcal {B}})} . The discriminator's strategy set is the set of measurable functions of type D : Ω → R {\displaystyle D:\Omega \to \mathbb {R} } with bounded Lipschitz-norm: ‖ D ‖ L ≤ K {\displaystyle \|D\|_{L}\leq K} . The Wasserstein GAN game is a zero-sum game, with objective function L W G A N ( μ G , D ) := E x ∼ μ G [ D ( x ) ] − E x ∼ μ r e f [ D ( x ) ] . {\displaystyle L_{WGAN}(\mu _{G},D):=\mathbb {E} _{x\sim \mu _{G}}[D(x)]-\mathbb {E} _{x\sim \mu _{ref}}[D(x)].} The generator goes first, and the discriminator goes second. The generator aims to minimize the objective, and the discriminator aims to maximize the objective: min μ G max D L W G A N ( μ G , D ) . {\displaystyle \min _{\mu _{G}}\max _{D}L_{WGAN}(\mu _{G},D).} By the Kantorovich-Rubenstein duality, for any generator strategy μ G {\displaystyle \mu _{G}} , the optimal reply by the discriminator is D ∗ {\displaystyle D^{}} , such that L W G A N ( μ G , D ∗ ) = K ⋅ W 1 ( μ G , μ r e f ) . {\displaystyle L_{WGAN}(\mu _{G},D^{})=K\cdot W_{1}(\mu _{G},\mu _{ref}).} Consequently, if the discriminator is good, the generator would be constantly pushed to minimize W 1 ( μ G , μ r e f ) {\displaystyle W_{1}(\mu _{G},\mu _{ref})} , and the optimal strategy for the generator is just μ G = μ r e f {\displaystyle \mu _{G}=\mu _{ref}} , as it should. == Comparison with GAN == In the Wasserstein GAN game, the discriminator provides a better gradient than in the GAN game. Consider for example a game on the real line where both μ G {\displaystyle \mu _{G}} and μ r e f {\displaystyle \mu _{ref}} are Gaussian. Then the optimal Wasserstein critic D W G A N {\displaystyle D_{WGAN}} and the optimal GAN discriminator D {\displaystyle D} are plotted as below: For fixed discriminator, the generator needs to minimize the following objectives: For GAN, E x ∼ μ G [ ln ( 1 − D ( x ) ) ] {\displaystyle \mathbb {E} _{x\sim \mu _{G}}[\ln(1-D(x))]} . For Wasserstein GAN, E x ∼ μ G [ D W G A N ( x ) ] {\displaystyle \mathbb {E} _{x\sim \mu _{G}}[D_{WGAN}(x)]} . Let μ G {\displaystyle \mu _{G}} be parametrized by θ {\displaystyle \theta } , then we can perform stochastic gradient descent by using two unbiased estimators of the gradient: ∇ θ E x ∼ μ G [ ln ( 1 − D ( x ) ) ] = E x ∼ μ G [ ln ( 1 − D ( x ) ) ⋅ ∇ θ ln ρ μ G ( x ) ] {\displaystyle \nabla _{\theta }\mathbb {E} _{x\sim \mu _{G}}[\ln(1-D(x))]=\mathbb {E} _{x\sim \mu _{G}}[\ln(1-D(x))\cdot \nabla _{\theta }\ln \rho _{\mu _{G}}(x)]} ∇ θ E x ∼ μ G [ D W G A N ( x ) ] = E x ∼ μ G [ D W G A N ( x ) ⋅ ∇ θ ln ρ μ G ( x ) ] {\displaystyle \nabla _{\theta }\mathbb {E} _{x\sim \mu _{G}}[D_{WGAN}(x)]=\mathbb {E} _{x\sim \mu _{G}}[D_{WGAN}(x)\cdot \nabla _{\theta }\ln \rho _{\mu _{G}}(x)]} where we used the reparameterization trick. As shown, the generator in GAN is motivated to let its μ G {\displaystyle \mu _{G}} "slide down the peak" of ln ( 1 − D ( x ) ) {\displaystyle \ln(1-D(x))} . Similarly for the generator in Wasserstein GAN. For Wasserstein GAN, D W G A N {\displaystyle D_{WGAN}} has gradient 1 almost everywhere, while for GAN, ln ( 1 − D ) {\displaystyle \ln(1-D)} has flat gradient in the middle, and steep gradient elsewhere. As a result, the variance for the estimator in GAN is usually much larger than that in Wasserstein GAN. See also Figure 3 of. The problem with D J S {\displaystyle D_{JS}} is much more severe in actual machine learning situations. Consider training a GAN to generate ImageNet, a collection of photos of size 256-by-256. The space of all such photos is R 256 2 {\displaystyle \mathbb {R} ^{256^{2}}} , and the distribution of ImageNet pictures, μ r e f {\displaystyle \mu _{ref}} , concentrates on a manifold of much lower dimension in it. Consequently, any generator strategy μ G {\displaystyle \mu _{G}} would almost surely be entirely disjoint from μ r e f {\displaystyle \mu _{ref}} , making D J S ( μ G ‖ μ r e f ) = + ∞ {\displaystyle D_{JS}(\mu _{G}\|\mu _{ref})=+\infty } . Thus, a good discriminator can almost perfectly distinguish μ r e f {\displaystyle \mu _{ref}} from μ G {\displaystyle \mu _{G}} , as well as any μ G ′ {\displaystyle \mu _{G}'} close to μ G {\displaystyle \mu _{G}} . Thus, the gradient ∇ μ G L ( μ G , D ) ≈ 0 {\displaystyle \nabla _{\mu _{G}}L(\mu _{G},D)\approx 0} , creating no learning signal for the generator. Detailed theorems can be found in. == Training Wasserstein GANs == Training the generator in Wasserstein GAN is just gradient descent, the same as in GAN (or most deep learning methods), but training the discriminator is different, as the discriminator is now restricted to have bounded Lipschitz norm. There are several methods for this. === Upper-bounding the Lipschitz norm === Let the discriminator function D {\displaystyle D} to be implemented by a multilayer perceptron: D = D n ∘ D n − 1 ∘ ⋯ ∘ D 1 {\displaystyle D=D_{n}\circ D_{n-1}\circ \cdots \circ D_{1}} where D i ( x ) = h ( W i x ) {\displaystyle D_{i}(x)=h(W_
HTK Limited
HTK Limited is a software-as-a-service company that provides mobile phone messaging and IVR services. Founded in 1996, HTK is headquartered in Ipswich, Suffolk, UK. HTK provide mass notification services. Specifically, the "Police Direct" messaging service to Suffolk and Norfolk Constabularies. In 2010 the HTK Horizon SaaS platform was selected by the Scottish Environment Protection Agency (SEPA) for their Floodline Warnings Direct service. == History == HTK was founded in 1996 by Marlon Bowser and Adrian Gregory and from the outset focused on what has now become commonly known as Software-as-a-Service. in 2004, according to the Deloitte Fast 50 (UK), HTK was the 17th fastest growing company in the East of England. In 2005 The Times listed HTK 65th nationally and 4th in the East of England in the Sunday Times & Microsoft "Tech Track 100" awards. In 2009 the company was approved as a supplier to UK Government under a new framework agreement. In 2010 HTK launched version 2.2 of its Horizon platform, with a feature set that signals a shift from mass notification into the customer service automation market.
Korpusomat
Korpusomat - a tool for creating and searching electronic language corpora, created at the Institute of Computer Science of the Polish Academy of Sciences. Korpusomat is a fourth generation corpus tool. It is a web application, which eliminates the need to store data sets on the user's own computer. The corpus is created either by adding text files from the local drive (in any language and format), or by indicating websites from which texts are to be downloaded. Then, the corpus is annotated automatically on several levels: morphosyntantic, named entities recognition (e.g. geographical names or people) and partial syntantic information (which also allows for the visualization of dependency trees). The finished corpus can be edited, shared with other users, and searched. There are also a number of functions offering statistical summaries of the collected texts