State–action–reward–state–action (SARSA) is an algorithm for learning a Markov decision process policy, used in the reinforcement learning area of machine learning. It was proposed by Rummery and Niranjan in a technical note with the name "Modified Connectionist Q-Learning" (MCQ-L). The alternative name SARSA, proposed by Rich Sutton, was only mentioned as a footnote. This name reflects the fact that the main function for updating the Q-value depends on the current state of the agent "S1", the action the agent chooses "A1", the reward "R2" the agent gets for choosing this action, the state "S2" that the agent enters after taking that action, and finally the next action "A2" the agent chooses in its new state. The acronym for the quintuple (St, At, Rt+1, St+1, At+1) is SARSA. Some authors use a slightly different convention and write the quintuple (St, At, Rt, St+1, At+1), depending on which time step the reward is formally assigned. The rest of the article uses the former convention. == Algorithm == Q new ( S t , A t ) ← ( 1 − α ) Q ( S t , A t ) + α [ R t + 1 + γ Q ( S t + 1 , A t + 1 ) ] {\displaystyle Q^{\textrm {new}}(S_{t},A_{t})\leftarrow (1-\alpha )Q(S_{t},A_{t})+\alpha \,[R_{t+1}+\gamma \,Q(S_{t+1},A_{t+1})]} A SARSA agent interacts with the environment and updates the policy based on actions taken, hence this is known as an on-policy learning algorithm. The Q value for a state-action is updated by an error, adjusted by the learning rate α. Q values represent the possible reward received in the next time step for taking action a in state s, plus the discounted future reward received from the next state-action observation. Watkin's Q-learning updates an estimate of the optimal state-action value function Q ∗ {\displaystyle Q^{}} based on the maximum reward of available actions. While SARSA learns the Q values associated with taking the policy it follows itself, Watkin's Q-learning learns the Q values associated with taking the optimal policy while following an exploration/exploitation policy. Some optimizations of Watkin's Q-learning may be applied to SARSA. == Hyperparameters == === Learning rate (alpha) === The learning rate determines to what extent newly acquired information overrides old information. A factor of 0 will make the agent not learn anything, while a factor of 1 would make the agent consider only the most recent information. === Discount factor (gamma) === The discount factor determines the importance of future rewards. A discount factor of 0 makes the agent "opportunistic", or "myopic", e.g., by only considering current rewards, while a factor approaching 1 will make it strive for a long-term high reward. If the discount factor meets or exceeds 1, the Q {\displaystyle Q} values may diverge. === Initial conditions (Q(S0, A0)) === Since SARSA is an iterative algorithm, it implicitly assumes an initial condition before the first update occurs. A high (infinite) initial value, also known as "optimistic initial conditions", can encourage exploration: no matter what action takes place, the update rule causes it to have higher values than the other alternative, thus increasing their choice probability. In 2013 it was suggested that the first reward r {\displaystyle r} could be used to reset the initial conditions. According to this idea, the first time an action is taken the reward is used to set the value of Q {\displaystyle Q} . This allows immediate learning in case of fixed deterministic rewards. This resetting-of-initial-conditions (RIC) approach seems to be consistent with human behavior in repeated binary choice experiments.
FastTrack Automation Studio
FastTrack Automation Studio (formerly known as FastTrack Scripting Host), often referred to as just FastTrack, is a scripting language for Windows IT System Administrators. The product’s goal is to handle any kind of scripting that might be required to automate processes with Microsoft Windows networks. == Manufacturer == FastTrack is produced by FastTrack Software, which is headquartered in Aalborg, Denmark. The product is promoted by the manufacturer as a one-stop shop for Windows script writers and its development paradigm is “one operation = one script line”. Script writers use a purpose-built editor to create scripts, inserting script lines via menus, drag’n drop, or simply typing them in. Scripts may be used out of the box, created from scratch, imported from forums or other users, or customized from product documentation. == Types of scripts == Simple scripts include: Outlook Signatures Login scripts Backup and replication scripts Inventory and asset management Automated Windows OS installation and deployment Automated application software deployment Active Directory scripts More advanced scripts include: SCCM task sequences Citrix ICA and RDP Clients built-in Deploying applications to server farms Deploying GPO MSI files SQL Server scripts == Basic structure == Under the hood, scripts comprise commands, functions, collections, and conditions. When a script is executed these components are converted into many lines of C# code, sometimes hundreds of lines, depending on the particular script operation. Scripts can be compiled into EXE files or MSI packages and treated as standalone Windows applications. == History == FastTrack Scripting Host (FastTrack) was first developed around 2006 to ease the administration burden of IT System Administrators on Windows networks. === Product idea === The idea for the product came from founder and President of FastTrack Software, Lars Pedersen, who has a background in systems administration. Previously with Telenor, Denmark’s major telephone company, Pedersen performed various roles in systems administration, programming and web development. He also worked as a consultant and developer on several major projects at various companies in Europe. Dissatisfied from his own experiences and frustrations administering Windows networks, Pederson looked for a way to make life easier for system administrators. In particular, he wanted something that could minimize the amount of time needed each day to perform routine and mundane tasks, which was a waste of time and expertise that should have been committed to other projects. === Development === Leading a small team of developers, Pedersen developed FastTrack Scripting Host to simplify and automate the routine tasks of system administrators. The resulting product is definitely a scripting language, but it can be used intuitively like a programming language, without requiring users to learn syntax or other concepts typically associated with programming languages. === Marketing === In April 2010, FastTrack Software entered into an agreement with Binary Research International Archived 2008-10-15 at the Wayback Machine, based in the city of Milwaukee, United States to market and sell the product globally. === Awards === FSH received a Windows IT Pro Community Choice award in 2012. == Versions == The first version was produced in June 2006 and contained 51 components, which are the commands, functions, conditions and collections making up FastTrack. The following table summarizes dates and components for major releases. Companies and organizations such as NOAA, Kawasaki, and Goodyear have used and implemented the FastTrack Scripting Host. == Comparison with other scripting software == FastTrack Scripting Host Kixtart PowerShell ScriptLogic VBScript
Umbrella review
In medical research, an umbrella review is a review of systematic reviews or meta-analyses. They may also be called overviews of reviews, reviews of reviews, summaries of systematic reviews, or syntheses of reviews. Umbrella reviews are among the highest levels of evidence currently available in medicine. By summarizing information from multiple overview articles, umbrella reviews make it easier to review the evidence and allow for comparison of results between each of the individual reviews. Umbrella reviews may address a broader question than a typical review, such as discussing multiple different treatment comparisons instead of only one. They are especially useful for developing guidelines and clinical practice, and when comparing competing interventions.
Sikidy
Sikidy is a form of algebraic geomancy practiced by Malagasy peoples in Madagascar. It involves algorithmic operations performed on random data generated from tree seeds, which are ritually arranged in a tableau called a toetry and divinely interpreted after being mathematically operated on. Columns of seeds, designated "slaves" or "princes" belonging to respective "lands" for each, interact symbolically to express vintana ('fate') in the interpretation of the diviner. The diviner also prescribes solutions to problems and ways to avoid fated misfortune, often involving a sacrifice. The centuries-old practice derives from Islamic influence brought to the island by medieval Arab traders. The sikidy is consulted for a range of divinatory questions pertaining to fate and the future, including identifying sources of and rectifying misfortune, reading the fate of newborns, and planning annual migrations. The mathematics of sikidy involves Boolean algebra, symbolic logic and parity. == History == The practice is several centuries old, and is influenced by Arab geomantic traditions of Arab Muslim traders on the island. Most writers link the origins of sikidy to the "sea-going trade involving the southwest coast of India, the Persian Gulf, and the east coast of Africa in the 9th or 10th century C.E." Stephen Ellis and Solofo Randrianja describe sikidy as "probably one of the oldest components of Malagasy culture", writing that it most likely the product of an indigenous divinatory art later influenced by Islamic practice. Umar H. D. Danfulani writes that the integration of Arabic divination into indigenous divination is "clearly demonstrated" in Madagascar, where the Arabic astrological system was adapted to the indigenous agricultural system and meshed with Malagasy lunar months by "adapting indigenous months, volana, to the astrological months, vintana". Danfulani also describes the concepts in sikidy of "houses" (lands) and "kings in their houses" as retained from medieval Arabic astrology. Chemillier et al. say the practice's spread across Madagascar likely originated with the southeastern Antemoro people, among whom Arab influence was the strongest. Though the etymology of sikidy is unknown, it has been posited that the word derives from the Arabic sichr ('incantation' or 'charm'). Sikidy was of central importance to pre-Christian Malagasy religion, with one practitioner quoted in 1892 as calling sikidy "the Bible of our ancestors". A missionary report from 1616 describes one form of sikidy using tamarind seeds, and another using fingered markings in the sand. The early colonial French governor of Madagascar Étienne de Flacourt documented sikidy in the mid-17th century: Matatane country in southeastern Madagascar [...] where the Antemoro [...] live was a center of astrological study as early as the fourteenth century [...]. This area was also the site of early Arab settlements, although strict Islamic observances were lost centuries ago [...]. Historical evidence shows that Antemoro diviners, bearers of the astrological system, infiltrated nearly all the ancient kingdoms of Madagascar beginning in the sixteenth century. [...] Today, although many persons claim to be ombiasy [diviners], only the Antemoro diviners are considered true professionals. The area is still a famous place of learning where specialists go for training and then return to their home communities with a certain body of knowledge. Now we can better understand the degree of similarity of divination forms found throughout Madagascar. For centuries Matitanana has remained a training center for diviners who have migrated widely, usually attaining important positions in their home communities and with various royal families. Comparison of contemporary rites with centuries-old texts show that sikidy has been remarkably unchanged throughout its history. The "infiltration" of Malagasy kingdoms by Antemoro diviners, and Matitanana's role as a place for astrological and divinatory learning, help to explain the relatively uniform practicing of sikidy across Madagascar. Chemallier et al. write that the mathematical construction of the arrangement of seeds is procedurally consistent across all of Madagascar, with variations in practice between groups and regions being limited to more minor aspects, such as the alignment of figures according to cardinal directions. One exception is the simplified Merina sikidy joria. === Origin myths === Mythic tradition relating to the origin of sikidy "links [the practice] both to the return by walking on water of Arab ancestors who had intermarried with Malagasy but then left, and to the names of the days of the week" and holds that the art was supernaturally communicated to the ancestors, with Zanahary (the supreme deity of Malagasy religion) giving it to Ranakandriana, who then gave it to a line of diviners (Ranakandriana to Ramanitralanana to Rabibi-andrano to Andriambavi-maitso (who was a woman) to Andriam-bavi-nosy), the last of whom terminated the monopoly by giving it to the people, declaring: "Behold, I give you the sikidy, of which you may inquire what offerings you should present in order to obtain blessings; and what expiation you should make so as to avert evils, when any are ill or under apprehension of some future calamity". A mythic anecdote of Ranakandriana says that two men observed him one day playing in the sand. In fact he was practicing a form of sikidy worked in sand called sikidy alanana. The two men seized him, and Ranakandriana promised that he would teach them something if they released him. They agreed, and Ranakandriana taught them in depth how to work the sikidy. The two men then went to their chief and told him that they could tell him "the past and the future—what was good and what was bad—what increased and what diminished." The chief asked them to tell him how he could obtain plenty of cattle. The two men worked their sikidy and told the chief to kill all of his bulls, and that "great numbers would come to him" on the following Friday. The chieftain, doubting, asked what would happen if their prediction didn't come true, and the two men promised they would pay with their lives. The chief agreed and killed his bulls. On Thursday, thinking he'd been duped, he prematurely killed the first man of the two who'd told him about the divinatory art. On Friday, however, "vast herds" came amidst heavy rain, actually filling an immense plain in their crowd. The chieftain lamented the mpisikidy's wrongful execution and ordered for him a pompous funeral. The chieftain took the second man as his close adviser and friend, and trusted the sikidy forever afterwards. The British missionary William Ellis recorded in 1839 two idiomatic expressions used in Madagascar that come from this story: "Tsy mahandry andro Zoma" (lit. 'He cannot wait 'til Friday') is said of someone extremely impatient, and heavy rainshowers falling in rapid succession are called "sese omby" (lit. 'a crowding together of cattle'). == Rites and arrangement of seeds == The divination is performed by a practitioner called an mpisikidy, ny màsina (lit. 'sacred one'), ombiasy, or ambiàsa (derived from the Arabic anbia, meaning 'prophet') who guides the client through the process and interprets the results in the context of the client's inquiries and desires. As part of an mpisikidy's formal initiation into the art, which includes a long period of apprenticeship, the initiate (called a mianatsy) must gather 124 and 200 fàno (Entada sp.) or kily (tamarind) tree seeds for his subsequent ritual use in sikidy. Raymond Decary writes that, at least among the Sakalava, a man must be 40 years old before learning and practicing sikidy, or he risks death. Before beginning to study, a student practitioner must make incisions at the tips of his index finger, his middle finger, and his tongue, and put within the incisions a paste containing red pepper and crushed wasp. This paste impregnates the fingers that will move the seeds of the sikidy and the tongue that will speak their revelations with the power to decipher the sikidy. Once this is done, he leaves at dawn to search for a fano (Entada chrysostachys) tree. Upon finding it, he throws his spear at its branches, shaking the tree and causing its large seed pods to fall. During this act, some initiates say: "When you were on the steep peak and in the dense forest, on you the crabs climbed, from you the crocodiles made their bed, with their paws the birds trod on you. Whether you are suspended in the trees or buried, you are never dried up nor rotten." In his study (written in 1941 and revised in 1948), Decary reported that the salary paid by a mianatsy to his master is "not very high": up to five francs, plus a red rooster's feather. The mpisikidy ritually arranges his seeds into a sixteen-column table consisting of four columns of randomly-generated data (representing fate) and eight columns of data derived from logical ope
Operational historian
In manufacturing, an operational historian is a time-series database application that is developed for operational process data. Historian software is often embedded or used in conjunction with standard DCS and PLC control systems to provide enhanced data capture, validation, compression, and aggregation capabilities. Historians have been deployed in almost every industry and contribute to functions such as supervisory control, performance monitoring, quality assurance, and, more recently, machine learning applications which can learn from vast quantities of historical data. These systems were originally developed to capture instrumentation and control data, which led many to use the term "tag" for a stream of process data, referring to the physical "tags" which had been placed on instrumentation for manually capturing data. Raw data may be accessed via OPC HDA, SQL, or REST API interfaces. == Operational Support == Operational historians are typically used within the manufacturing facility by engineers and operators for supervisory functions and analysis. An operational historian will typically capture all instrumentation and control data, whereas an enterprise historian that is deployed to support business functions will capture only a subset of the plant data. Typically, these applications offer data access through dedicated APIs (Application Programming Interfaces) and SDKs (Software Development Kits) which offer high-performance read and write operations. These operate through vendor-specific or custom applications. Front-end tools for trending process data over time are the most common interfaces to these databases. Because these applications are typically deployed next to or near the source of their process data, they are often marketed and sold as 'real-time database systems.' This distinction varies among vendors, who often have to make tradeoffs in performance between data capture and presentation, and application and analysis functionality. The following is a list of typical challenges for operational historians: data collection from instrumentation and controls storage and archiving of very large volumes of data organization of data in the form of "tags" or "points" limiting of monitoring (alarms) and validation aggregation and interpolation manual data entry (MDE) == Data access == As opposed to enterprise historians, the data access layer in the operational historian is designed to offer sophisticated data fetching modes without complex information analysis facilities. The following settings are typically available for data access operations: Data scope (single point or tag, history based on time range, history based on sample count) Request modes (raw data, last-known value, aggregation, interpolation) Sampling (single point, all points without sampling, all points with interval sampling) Data omission (based on the sample quality, based on the sample value, based on the count) Even though the operational historians are rarely relational database management systems, they often offer SQL-based interfaces to query the database. In most of such implementations, the dialect does not follow the SQL standard in order to provide syntax for specifying data access operations parameters.
Security switch
A security switch is a hardware device designed to protect computers, laptops, smartphones and similar devices from unauthorized access or operation, distinct from a virtual security switch which offers software protection. Security switches should be operated by an authorized user only; for this reason, it should be isolated from other devices, in order to prevent unauthorized access, and it should not be possible to bypass it, in order to prevent malicious manipulation. The primary purpose of a security switch is to provide protection against surveillance, eavesdropping, malware, spyware, and theft of digital devices. Unlike other protections or techniques, a security switch can provide protection even if security has already been breached, since it does not have any access from other components and is not accessible by software. It can additionally disconnect or block peripheral devices, and perform "man in the middle" operations. A security switch can be used for human presence detection since it can only be initiated by a human operator. It can also be used as a firewall. == Types == === Hardware kill switch === A hardware kill switch (HKS) is a physical switch that cuts the signal or power line to the device or disable the chip running them. == Examples == A cellphone is compromised by malicious software, and the device initiates video and audio recording. When the user activates the “prevent capture of audio/video” mode of the security switch, that either physically disconnects or cut the power to the microphone and the camera, which stops the recording. A laptop that has an embedded security switch is stolen. The security switch detects a lack of communication from a specific external source for 12 hours, and responds by disconnecting the screen, keyboard and other key components, rendering the laptop useless, with no possibility of recovery, even with a full format. A user wishes to prevent tracking of their location. The user then activates geolocation protection and the security switch disables all GPS communication, eliminating the possibility of tracking the device's location. A user desires to eliminate the possibility of their PIN being copied from their smartphone. They can activate the secure input function, causing the security switch to disconnect the touch screen from the operating system, so input signals are not available to any devices except the switch. A security switch performs scheduled monitoring and finds that a program is attempting to download malicious content from the internet. It then activates internet security function and disables internet access, interrupting the download. If laptop software is compromised by air-gap malware, the user may activate the security switch and disconnect the speaker and microphone, so it can not establish communication with the device. == History == Google started to work on a hardware kill switch for AI in 2016. In 2019, Apple, and Google, along with a handful of smaller players, are designing “kill switches” that cut the power to the microphones or cameras in their devices. Googles first product that implemented this is Nest Hub Max. Hardware kill switches are already available and widely tested on the PinePhone, Librem, Shiftphone, to cut power to the input peripherals (microphone, camera) but also the network connectivity modules (wifi, cellular network).
Relational data stream management system
A relational data stream management system (RDSMS) is a distributed, in-memory data stream management system (DSMS) that is designed to use standards-compliant SQL queries to process unstructured and structured data streams in real-time. Unlike SQL queries executed in a traditional RDBMS, which return a result and exit, SQL queries executed in a RDSMS do not exit, generating results continuously as new data become available. Continuous SQL queries in a RDSMS use the SQL Window function to analyze, join and aggregate data streams over fixed or sliding windows. Windows can be specified as time-based or row-based. == RDSMS SQL Query Examples == Continuous SQL queries in a RDSMS conform to the ANSI SQL standards. The most common RDSMS SQL query is performed with the declarative SELECT statement. A continuous SQL SELECT operates on data across one or more data streams, with optional keywords and clauses that include FROM with an optional JOIN subclause to specify the rules for joining multiple data streams, the WHERE clause and comparison predicate to restrict the records returned by the query, GROUP BY to project streams with common values into a smaller set, HAVING to filter records resulting from a GROUP BY, and ORDER BY to sort the results. The following is an example of a continuous data stream aggregation using a SELECT query that aggregates a sensor stream from a weather monitoring station. The SELECTquery aggregates the minimum, maximum and average temperature values over a one-second time period, returning a continuous stream of aggregated results at one second intervals. RDSMS SQL queries also operate on data streams over time or row-based windows. The following example shows a second continuous SQL query using the WINDOW clause with a one-second duration. The WINDOW clause changes the behavior of the query, to output a result for each new record as it arrives. Hence the output is a stream of incrementally updated results with zero result latency.