Stefan Schaal

Stefan Schaal (born 1961) is a German-American computer scientist specializing in robotics, machine learning, autonomous systems, and computational neuroscience. == Education and career == Schaal was born in Frankfurt am Main in Germany, Schaal grew up in the North Bavarian town of Nürnberg. After graduating from school, he served in the German army in the Ski Patrol Division of Bad Reichenhall, where he honorably discharged with the rank of a Lieutenant. Schaal studied mechanical engineering at the Technical University of Munich, graduating in 1987 with a Diploma degree (summa cum laude). Subsequently, Schaal did his Ph.D. in computer aided design and artificial intelligence at the Technical University of Munich and the Massachusetts Institute of Technology, receiving his Ph.D. in 1991 (Summa Cum Laude) under Klaus Ehrlenspiel. In 1991, Schaal was a Postdoctoral Fellow at the Department and Brain and Cognitive Science and the Artificial Intelligence Lab at the Massachusetts Institute of Technology, funded by the Alexander von Humboldt Foundation and the German Academic Scholarship Foundation. Starting from 1992, he became an invited researcher at the ATR Computational Neuroscience Labs in Japan, where he created a robotics lab focusing on biological principles of motor control and learning. In 1994, Schaal moved to the Georgia Institute of Technology as an adjunct assistant professor, and also held the same rank at the Pennsylvania State University. In 1996, Schaal assumed a group leader position in the ERATO Kawato Dynamic Brain Project in Japan. Schaal joined the University of Southern California (USC) in 1997, where he advanced from the ranks of assistant professor, to associate professor, to full professor. In 2009, Schaal became a founder in defining and creating the Max Planck Institute for Intelligent Systems in Tübingen and Stuttgart, Germany, an institute focusing on principles of perception-action-learning systems in synthetic intelligence. In 2012, Schaal founded the Autonomous Motion Department (AMD) at this institute, while maintaining a partial appointment at USC. Stefan Schaal joined Google X as lead of a robotics research team in late 2018. == Research == Stefan Schaal's interests focus on autonomous perception-action-learning systems, in particular anthropomorphic robotic systems. He works on topics of machine learning for control, control theory, computational neuroscience for neuromotor control, experimental robotics, reinforcement learning, artificial intelligence, and nonlinear dynamical systems. Stefan has co-authored more than 400 publications in top conferences and journals, and served as organizer on various top conferences in machine learning and robotics. He has received numerous best paper awards and honors in his scientific community. Stefan Schaal has been noted as one of the five leaders in robotics in 2011, and among the top robotics experts in the world. == Controversy == In 2018, the German newsjournal Der Spiegel published an article reporting on his double affiliation with USC and the Max-Planck Society, both with full salaries, which was apparently unknown to either party. Schaal rejected the allegations, but was forced to leave his position at the Max Planck Institute.

Cloud-native computing

Cloud native computing is an approach in software development that utilizes cloud computing to "build and run scalable applications in modern, dynamic environments such as public, private, and hybrid clouds". These technologies, such as containers, microservices, serverless functions, cloud native processors and immutable infrastructure, deployed via declarative code are common elements of this architectural style. Cloud native technologies focus on minimizing users' operational burden. Cloud native techniques "enable loosely coupled systems that are resilient, manageable, and observable. Combined with robust automation, they allow engineers to make high-impact changes frequently and predictably with minimal toil." This independence contributes to the overall resilience of the system, as issues in one area do not necessarily cripple the entire application. Additionally, such systems are easier to manage, and monitor, given their modular nature, which simplifies tracking performance and identifying issues. Frequently, cloud-native applications are built as a set of microservices that run in Open Container Initiative compliant containers, such as Containerd, and may be orchestrated in Kubernetes and managed and deployed using DevOps and Git CI workflows (although there is a large amount of competing open source that supports cloud-native development). The advantage of using containers is the ability to package all software needed to execute into one executable package. The container runs in a virtualized environment, which isolates the contained application from its environment.

Certifying algorithm

In theoretical computer science, a certifying algorithm is an algorithm that outputs, together with a solution to the problem it solves, a proof that the solution is correct. A certifying algorithm is said to be efficient if the combined runtime of the algorithm and a proof checker is slower by at most a constant factor than the best known non-certifying algorithm for the same problem. The proof produced by a certifying algorithm should be in some sense simpler than the algorithm itself, for otherwise any algorithm could be considered certifying (with its output verified by running the same algorithm again). Sometimes this is formalized by requiring that a verification of the proof take less time than the original algorithm, while for other problems (in particular those for which the solution can be found in linear time) simplicity of the output proof is considered in a less formal sense. For instance, the validity of the output proof may be more apparent to human users than the correctness of the algorithm, or a checker for the proof may be more amenable to formal verification. Implementations of certifying algorithms that also include a checker for the proof generated by the algorithm may be considered to be more reliable than non-certifying algorithms. For, whenever the algorithm is run, one of three things happens: it produces a correct output (the desired case), it detects a bug in the algorithm or its implication (undesired, but generally preferable to continuing without detecting the bug), or both the algorithm and the checker are faulty in a way that masks the bug and prevents it from being detected (undesired, but unlikely as it depends on the existence of two independent bugs). == Examples == Many examples of problems with checkable algorithms come from graph theory. For instance, a classical algorithm for testing whether a graph is bipartite would simply output a Boolean value: true if the graph is bipartite, false otherwise. In contrast, a certifying algorithm might output a 2-coloring of the graph in the case that it is bipartite, or a cycle of odd length if it is not. Any graph is bipartite if and only if it can be 2-colored, and non-bipartite if and only if it contains an odd cycle. Both checking whether a 2-coloring is valid and checking whether a given odd-length sequence of vertices is a cycle may be performed more simply than testing bipartiteness. Analogously, it is possible to test whether a given directed graph is acyclic by a certifying algorithm that outputs either a topological order or a directed cycle. It is possible to test whether an undirected graph is a chordal graph by a certifying algorithm that outputs either an elimination ordering (an ordering of all vertices such that, for every vertex, the neighbors that are later in the ordering form a clique) or a chordless cycle. And it is possible to test whether a graph is planar by a certifying algorithm that outputs either a planar embedding or a Kuratowski subgraph. The extended Euclidean algorithm for the greatest common divisor of two integers x and y is certifying: it outputs three integers g (the divisor), a, and b, such that ax + by = g. This equation can only be true of multiples of the greatest common divisor, so testing that g is the greatest common divisor may be performed by checking that g divides both x and y and that this equation is correct.

Algorithm IMED

In multi-armed bandit problems, IMED (for Indexed Minimum Empirical Divergence) is an algorithm developed in 2015 by Junya Honda and Akimichi Takemura. It is the first algorithm proved to be asymptotically optimal respect to the problem-dependant Lai–Robbins lower bound for distributions in ( − ∞ , 1 ] {\displaystyle (-\infty ,1]} . == Multi-armed bandit problem == The Multi-armed bandit problem is a sequential game where one player has to choose at each turn between K {\displaystyle K} actions (arms). Behind every arm a {\displaystyle a} there is an unknown distribution ν a {\displaystyle \nu _{a}} that lies in a set D {\displaystyle {\mathcal {D}}} known by the player (for example, D {\displaystyle {\mathcal {D}}} can be the set of Gaussian distributions or Bernoulli distributions). At each turn t {\displaystyle t} the player chooses (pulls) an arm a t {\displaystyle a_{t}} , he then gets an observation X t {\displaystyle X_{t}} of the distribution ν a t {\displaystyle \nu _{a_{t}}} . === Regret minimization === The goal is to minimize the regret at time T {\displaystyle T} that is defined as R T := ∑ a = 1 K Δ a E [ N a ( T ) ] {\displaystyle R_{T}:=\sum _{a=1}^{K}\Delta _{a}\mathbb {E} [N_{a}(T)]} where μ a := E [ ν a ] {\displaystyle \mu _{a}:=\mathbb {E} [\nu _{a}]} is the mean of arm a {\displaystyle a} μ ∗ := max a μ a {\displaystyle \mu ^{}:=\max _{a}\mu _{a}} is the highest mean Δ a := μ ∗ − μ a {\displaystyle \Delta _{a}:=\mu ^{}-\mu _{a}} N a ( t ) {\displaystyle N_{a}(t)} is the number of pulls of arm a {\displaystyle a} up to turn t {\displaystyle t} The player has to find an algorithm that chooses at each turn t {\displaystyle t} which arm to pull based on the previous actions and observations ( a s , X s ) s < t {\displaystyle (a_{s},X_{s})_{s μ } {\displaystyle {\mathcal {K}}_{inf}(\nu ,\mu ,{\mathcal {D}}):=\inf \left\{\mathrm {KL} (\nu ,{\tilde {\nu }})\ |\ {\tilde {\nu }}\in {\mathcal {P}}([-\infty ,1]),\ \mathbb {E} [{\tilde {\nu }}]>\mu \right\}} K L {\displaystyle \mathrm {KL} } is the Kullback–Leibler divergence P ( [ − ∞ , 1 ] ) {\displaystyle {\mathcal {P}}([-\infty ,1])} is the set of distribution in [ − ∞ , 1 ] {\displaystyle [-\infty ,1]} ν ^ a ( t ) {\displaystyle {\hat {\nu }}_{a}(t)} is the empirical distribution of arm a {\displaystyle a} at turn t {\displaystyle t} μ ^ ∗ ( t ) {\displaystyle {\hat {\mu }}^{}(t)} is the highest empirical mean of turn t {\displaystyle t} Remark : For arms a {\displaystyle a} that verify μ ^ a ( t ) = μ ^ ∗ ( t ) {\displaystyle {\hat {\mu }}_{a}(t)={\hat {\mu }}^{}(t)} we have K i n f ( ν ^ a ( t ) , μ ^ ∗ ( t ) ) = 0 {\displaystyle K_{inf}({\hat {\nu }}_{a}(t),{\hat {\mu }}^{}(t))=0} . Then there index is equal to ln ⁡ ( N a ( t ) ) {\displaystyle \ln(N_{a}(t))} === Pseudocode === for each arm i do: n[i] ← 1; nu[i] ← None; mu[i] ← None for t from 1 to K do: select arm t observe reward r n[t] ← n[t] + 1 nu[t] ← update empirical distribution mu[t] ← update empirical mean for t from K+1 to T do: mu ← highest mu for each arm i do: scoreK[i] ← n[i] K_inf(nu[i],mu) scoreN[i] ← ln(n[i]) index[i] ← scoreK[i] + scoreN[i] select arm a with smallest index[a] observe reward r n[a] ← n[a] + 1 nu[a] ← update empirical distribution mu[a] ← update empirical mean == Theoretical results == In the multi-armed bandit problem we have the asymptotic Lai–Robbins lower bound asymptotic lower bound on regret. The algorithm IMED is the first algorithm that matches this lower bound for distribution in ( − ∞ , 1 ] {\displaystyle (-\infty ,1]} in the first order. If the distribution are also bounded then it also match the second order. It is the first algorithm that match the second under of this lower bound. === Lai–Robbins lower bound === In 1985 Lai and Robbins proved an asymptotic, problem-dependent lower bound on regret. In 2018, Aurelien Garivier, Pierre Menard and Gilles Stoltz proved a refined lower bound that gives the second order It states that for every consistent algorithm on the set P ( [ − ∞ , 1 ] ) {\displaystyle {\mathcal {P}}([-\infty ,1])} — that is, an algorithm for which, for every ( ν 1 , … , ν K ) ∈ P ( [ − ∞ , 1 ] ) K {\displaystyle (\nu _{1},\dots ,\nu _{K})\in {\mathcal {P}}([-\infty ,1])^{K}} , the regret R T {\displaystyle R_{T}} is subpolynomial (i.e. R T = o T → + ∞ ( T α ) {\displaystyle R_{T}=o_{T\to +\infty }(T^{\alpha })} for all α > 0 {\displaystyle \alpha >0} ) — we have: R T ≥ ( ∑ a : μ a < μ ∗ Δ a K inf ( ν a , μ ∗ ) ) ln ⁡ T − Ω T → + ∞ ( ln ⁡ ln ⁡ T ) . {\displaystyle R_{T}\geq \left(\sum _{a:\mu _{a}<\mu ^{}}{\frac {\Delta _{a}}{{\mathcal {K}}_{\inf }(\nu _{a},\mu ^{})}}\right)\ln T-\Omega _{T\to +\infty }(\ln \ln T).} This bound is asymptotic (as T → + ∞ {\displaystyle T\to +\infty } ) and gives a first-order lower bound of order ln ⁡ T {\displaystyle \ln T} with the optimal constant in front of it and the second order in − Ω ( ln ⁡ ln ⁡ T ) {\displaystyle -\Omega (\ln \ln T)} . === Regret bound for IMED === If the distribution of every arm a {\displaystyle a} is ( − ∞ , 1 ] {\displaystyle (-\infty ,1]} ( i.e. ν a ∈ P ( [ − ∞ , 1 ] ) ) {\displaystyle \nu _{a}\in {\mathcal {P}}([-\infty ,1]))} then the regret of the algorithm IMED verify R T ≤ ( ∑ a : μ a < μ ∗ Δ a K inf ( ν a , μ ∗ ) ) ln ⁡ T + O ( 1 ) {\displaystyle R_{T}\leq \left(\sum _{a:\mu _{a}<\mu ^{}}{\frac {\Delta _{a}}{{\mathcal {K}}_{\inf }(\nu _{a},\mu ^{})}}\right)\ln T+O(1)} If all the distribution ν a {\displaystyle \nu _{a}} are bounded then it exists a constant C > 0 {\displaystyle C>0} such that for T {\displaystyle T} large enough the regret of IMED is upper bounded by R T ≤ ( ∑ a : μ a < μ ∗ Δ a K inf ( ν a , μ ∗ ) ) ln ⁡ T − C ln ⁡ ln ⁡ T {\displaystyle R_{T}\leq \left(\sum _{a:\mu _{a}<\mu ^{}}{\frac {\Delta _{a}}{{\mathcal {K}}_{\inf }(\nu _{a},\mu ^{})}}\right)\ln T-C\ln \ln T} == Computation time == The algorithm only requiere to compute the K i n f {\displaystyle K_{inf}} for suboptimal arms who are pulled O ( ln ⁡ T ) {\displaystyle O(\ln T)} times, which make it a lot faster than KL-UCB. A faster version of IMED was developed in 2023 to make it even faster, using a Taylor development of the K i n f {\displaystyle K_{inf}} in the first order .

In-place algorithm

In computer science, an in-place algorithm is an algorithm that operates directly on the input data structure without requiring extra space proportional to the input size. In other words, it modifies the input in place, without creating a separate copy of the data structure. An algorithm which is not in-place is sometimes called not-in-place or out-of-place. In-place can have slightly different meanings. In its strictest form, the algorithm can only have a constant amount of extra space, counting everything including function calls and pointers. However, this form is very limited as simply having an index to a length n array requires O(log n) bits. More broadly, in-place means that the algorithm does not use extra space for manipulating the input but may require a small though non-constant extra space for its operation. Usually, this space is O(log n), though sometimes anything in o(n) is allowed. Note that space complexity also has varied choices in whether or not to count the index lengths as part of the space used. Often, the space complexity is given in terms of the number of indices or pointers needed, ignoring their length. In this article, we refer to total space complexity (DSPACE), counting pointer lengths. Therefore, the space requirements here have an extra log n factor compared to an analysis that ignores the lengths of indices and pointers. An algorithm may or may not count the output as part of its space usage. Since in-place algorithms usually overwrite their input with output, no additional space is needed. When writing the output to write-only memory or a stream, it may be more appropriate to only consider the working space of the algorithm. In theoretical applications such as log-space reductions, it is more typical to always ignore output space (in these cases it is more essential that the output is write-only). == Examples == Given an array a of n items, suppose we want an array that holds the same elements in reversed order and to dispose of the original. One seemingly simple way to do this is to create a new array of equal size, fill it with copies from a in the appropriate order and then delete a. function reverse(a[0..n - 1]) allocate b[0..n - 1] for i from 0 to n - 1 b[n − 1 − i] := a[i] return b Unfortunately, this requires O(n) extra space for having the arrays a and b available simultaneously. Also, allocation and deallocation are often slow operations. Since we no longer need a, we can instead overwrite it with its own reversal using this in-place algorithm which will only need constant number (2) of integers for the auxiliary variables i and tmp, no matter how large the array is. function reverse_in_place(a[0..n-1]) for i from 0 to floor((n-2)/2) tmp := a[i] a[i] := a[n − 1 − i] a[n − 1 − i] := tmp As another example, many sorting algorithms rearrange arrays into sorted order in-place, including: bubble sort, comb sort, selection sort, insertion sort, heapsort, and Shell sort. These algorithms require only a few pointers, so their space complexity is O(log n). Quicksort operates in-place on the data to be sorted. However, quicksort requires O(log n) stack space pointers to keep track of the subarrays in its divide and conquer strategy. Consequently, quicksort needs O(log2 n) additional space. Although this non-constant space technically takes quicksort out of the in-place category, quicksort and other algorithms needing only O(log n) additional pointers are usually considered in-place algorithms. Most selection algorithms are also in-place, although some considerably rearrange the input array in the process of finding the final, constant-sized result. Some text manipulation algorithms such as trim and reverse may be done in-place. == In computational complexity == In computational complexity theory, the strict definition of in-place algorithms includes all algorithms with O(1) space complexity, the class DSPACE(1). This class is very limited; it equals the regular languages. In fact, it does not even include any of the examples listed above. Algorithms are usually considered in L, the class of problems requiring O(log n) additional space, to be in-place. This class is more in line with the practical definition, as it allows numbers of size n as pointers or indices. This expanded definition still excludes quicksort, however, because of its recursive calls. Identifying the in-place algorithms with L has some interesting implications; for example, it means that there is a (rather complex) in-place algorithm to determine whether a path exists between two nodes in an undirected graph, a problem that requires O(n) extra space using typical algorithms such as depth-first search (a visited bit for each node). This in turn yields in-place algorithms for problems such as determining if a graph is bipartite or testing whether two graphs have the same number of connected components. == Role of randomness == In many cases, the space requirements of an algorithm can be drastically cut by using a randomized algorithm. For example, if one wishes to know if two vertices in a graph of n vertices are in the same connected component of the graph, there is no known simple, deterministic, in-place algorithm to determine this. However, if we simply start at one vertex and perform a random walk of about 20n3 steps, the chance that we will stumble across the other vertex provided that it is in the same component is very high. Similarly, there are simple randomized in-place algorithms for primality testing such as the Miller–Rabin primality test, and there are also simple in-place randomized factoring algorithms such as Pollard's rho algorithm. == In functional programming == Functional programming languages often discourage or do not support explicit in-place algorithms that overwrite data, since this is a type of side effect; instead, they only allow new data to be constructed. However, good functional language compilers will often recognize when an object very similar to an existing one is created and then the old one is thrown away, and will optimize this into a simple mutation "under the hood". Note that it is possible in principle to carefully construct in-place algorithms that do not modify data (unless the data is no longer being used), but this is rarely done in practice.

Hekaton (database)

Hekaton (also known as SQL Server In-Memory OLTP) is an in-memory database for OLTP workloads built into Microsoft SQL Server. Hekaton was designed in collaboration with Microsoft Research and was released in SQL Server 2014. Traditional RDBMS systems were designed when memory resources were expensive, and were optimized for disk storage. Hekaton is instead optimized for a working set stored entirely in main memory, but is still accessible via T-SQL like normal tables. It is fundamentally different from the "DBCC PINTABLE" feature in earlier SQL Server versions. Hekaton was announced at the Professional Association for SQL Server (PASS) conference 2012.

Savepoint

A savepoint is a way of implementing subtransactions (also known as nested transactions) within a relational database management system by indicating a point within a transaction that can be "rolled back to" without affecting any work done in the transaction before the savepoint was created. Multiple savepoints can exist within a single transaction. Savepoints are useful for implementing complex error recovery in database applications. If an error occurs in the midst of a multiple-statement transaction, the application may be able to recover from the error (by rolling back to a savepoint) without needing to abort the entire transaction. A savepoint can be declared by issuing a SAVEPOINT name statement. All changes made after a savepoint has been declared can be undone by issuing a ROLLBACK TO SAVEPOINT name command. Issuing RELEASE SAVEPOINT name will cause the named savepoint to be discarded, but will not otherwise affect anything. Issuing the commands ROLLBACK or COMMIT will also discard any savepoints created since the start of the main transaction. Savepoints are defined in the SQL standard and are supported by all established SQL relational databases, including PostgreSQL, Oracle Database, Microsoft SQL Server, MySQL, IBM Db2, SQLite (since 3.6.8), Firebird, H2 Database Engine, and Informix (since version 11.50xC3).