Mountain Car, a standard testing domain in Reinforcement learning, is a problem in which an under-powered car must drive up a steep hill. Since gravity is stronger than the car's engine, even at full throttle, the car cannot simply accelerate up the steep slope. The car is situated in a valley and must learn to leverage potential energy by driving up the opposite hill before the car is able to make it to the goal at the top of the rightmost hill. The domain has been used as a test bed in various reinforcement learning papers. == Introduction == The mountain car problem, although fairly simple, is commonly applied because it requires a reinforcement learning agent to learn on two continuous variables: position and velocity. For any given state (position and velocity) of the car, the agent is given the possibility of driving left, driving right, or not using the engine at all. In the standard version of the problem, the agent receives a negative reward at every time step when the goal is not reached; the agent has no information about the goal until an initial success. == History == The mountain car problem appeared first in Andrew Moore's PhD thesis (1990). It was later more strictly defined in Singh and Sutton's reinforcement learning paper with eligibility traces. The problem became more widely studied when Sutton and Barto added it to their book Reinforcement Learning: An Introduction (1998). Throughout the years many versions of the problem have been used, such as those which modify the reward function, termination condition, and the start state. == Techniques used to solve mountain car == Q-learning and similar techniques for mapping discrete states to discrete actions need to be extended to be able to deal with the continuous state space of the problem. Approaches often fall into one of two categories, state space discretization or function approximation. === Discretization === In this approach, two continuous state variables are pushed into discrete states by bucketing each continuous variable into multiple discrete states. This approach works with properly tuned parameters but a disadvantage is information gathered from one state is not used to evaluate another state. Tile coding can be used to improve discretization and involves continuous variables mapping into sets of buckets offset from one another. Each step of training has a wider impact on the value function approximation because when the offset grids are summed, the information is diffused. === Function approximation === Function approximation is another way to solve the mountain car. By choosing a set of basis functions beforehand, or by generating them as the car drives, the agent can approximate the value function at each state. Unlike the step-wise version of the value function created with discretization, function approximation can more cleanly estimate the true smooth function of the mountain car domain. === Eligibility traces === One aspect of the problem involves the delay of actual reward. The agent is not able to learn about the goal until a successful completion. Given a naive approach for each trial the car can only backup the reward of the goal slightly. This is a problem for naive discretization because each discrete state will only be backed up once, taking a larger number of episodes to learn the problem. This problem can be alleviated via the mechanism of eligibility traces, which will automatically backup the reward given to states before, dramatically increasing the speed of learning. Eligibility traces can be viewed as a bridge from temporal difference learning methods to Monte Carlo methods. == Technical details == The mountain car problem has undergone many iterations. This section focuses on the standard well-defined version from Sutton (2008). === State variables === Two-dimensional continuous state space. V e l o c i t y = ( − 0.07 , 0.07 ) {\displaystyle Velocity=(-0.07,0.07)} P o s i t i o n = ( − 1.2 , 0.6 ) {\displaystyle Position=(-1.2,0.6)} === Actions === One-dimensional discrete action space. m o t o r = ( l e f t , n e u t r a l , r i g h t ) {\displaystyle motor=(left,neutral,right)} === Reward === For every time step: r e w a r d = − 1 {\displaystyle reward=-1} === Update function === For every time step: A c t i o n = [ − 1 , 0 , 1 ] {\displaystyle Action=[-1,0,1]} V e l o c i t y = V e l o c i t y + ( A c t i o n ) ∗ 0.001 + cos ( 3 ∗ P o s i t i o n ) ∗ ( − 0.0025 ) {\displaystyle Velocity=Velocity+(Action)0.001+\cos(3Position)(-0.0025)} P o s i t i o n = P o s i t i o n + V e l o c i t y {\displaystyle Position=Position+Velocity} === Starting condition === Optionally, many implementations include randomness in both parameters to show better generalized learning. P o s i t i o n = − 0.5 {\displaystyle Position=-0.5} V e l o c i t y = 0.0 {\displaystyle Velocity=0.0} === Termination condition === End the simulation when: P o s i t i o n ≥ 0.6 {\displaystyle Position\geq 0.6} == Variations == There are many versions of the mountain car which deviate in different ways from the standard model. Variables that vary include but are not limited to changing the constants (gravity and steepness) of the problem so specific tuning for specific policies become irrelevant and altering the reward function to affect the agent's ability to learn in a different manner. An example is changing the reward to be equal to the distance from the goal, or changing the reward to zero everywhere and one at the goal. Additionally, a 3D mountain car can be used, with a 4D continuous state space.
Document mosaicing
Document mosaicing is a process that stitches multiple, overlapping snapshot images of a document together to produce one large, high resolution composite. The document is slid under a stationary, over-the-desk camera by hand until all parts of the document are snapshotted by the camera's field of view. As the document slid under the camera, all motion of the document is coarsely tracked by the vision system. The document is periodically snapshotted such that the successive snapshots are overlap by about 50%. The system then finds the overlapped pairs and stitches them together repeatedly until all pairs are stitched together as one piece of document. The document mosaicing can be divided into four main processes. Tracking Feature detecting Correspondences establishing Images mosaicing. == Tracking (simple correlation process) == In this process, the motion of the document slid under the camera is coarsely tracked by the system. Tracking is performed by a process called simple correlation process. In the first frame of snapshots, a small patch is extracted from the center of the image as a correlation template. The correlation process is performed in the four times size of the patch area of the next frame. The motion of the paper is indicated by the peak in the correlation function. The peak in the correlation function indicates the motion of the paper. The template is resampled from this frame and the tracking continues until the template reaches the edge of the document. After the template reaches the edge of the document, another snapshot is taken and the tracking process performs repeatedly until the whole document is imaged. The snapshots are stored in an ordered list to facilitate pairing the overlapped images in later processes. == Feature detecting for efficient matching == Feature detection is the process of finding the transformation that aligns one image with another. There are two main approaches for feature detection. Feature-based approach : Motion parameters are estimated from point correspondences. This approach is suitable for the case that there is plenty supply of stable and detectable features. Featureless approach : When the motion between the two images is small, the motion parameters are estimated using optical flow. On the other hand, when the motion between the two images is large, the motion parameters are estimated using generalised cross-correlation. However, this approach requires a computationally expensive resources. Each image is segmented into a hierarchy of columns, lines, and words to match the organised sets of features across images. Skew angle estimation and columns, lines and words finding are the examples of feature detection operations. === Skew angle estimation === Firstly, the angle that the rows of text make with the image raster lines (skew angle) is estimated. It is assumed to lie in the range of ±20°. A small patch of text in the image is selected randomly and then rotated in the range of ±20° until the variance of the pixel intensities of the patch summed along the raster lines is maximised. To ensure that the found skew angle is accurate, the document mosaic system performs calculation at many image patches and derive the final estimation by finding the average of the individual angles weighted by the variance of the pixel intensities of each patch. === Columns, lines and words finding === In this operation, the de-skewed document is intuitively segmented into a hierarchy of columns, lines and words. The sensitivity to illumination and page coloration of the de-skewed document can be removed by applying a Sobel operator to the de-skewed image and thresholding the output to obtain the binary gradient, de-skewed image. The operation can be roughly separated into 3 steps: column segmentation, line segmentation and word segmentation. Columns are easily segmented from the binary gradient, de-skewed images by summing pixels vertically. Baselines of each row are segmented in the same way as the column segmentation process but horizontally. Finally, individual words are segmented by applying the vertical process at each segmented row. These segmentations are important because the document mosaic is created by matching the lower right corners of words in overlapping images pair. Moreover, the segmentation operation can organize the list of images in the context of a hierarchy of rows and column reliably. The segmentation operation involves a considerable amount of summing in the binary gradient, de-skewed images, which done by construct a matrix of partial sums whose elements are given by p i y = ∑ u = 1 i ∑ v = 1 j b u v {\displaystyle p_{iy}=\sum _{u=1}^{i}\sum _{v=1}^{j}b_{uv}} The matrix of partial sums is calculated in one pass through the binary gradient, de-skewed image. ∑ u = u 1 u 2 ∑ v = v 1 v 2 b u v = p u 2 v 2 + p u 1 v 1 − p u 1 v 2 − p u 2 v 1 {\displaystyle \sum _{u=u_{1}}^{u_{2}}\sum _{v=v_{1}}^{v_{2}}b_{uv}=p_{u_{2}v_{2}}+p_{u_{1}v_{1}}-p_{u_{1}v_{2}}-p_{u_{2}v_{1}}} == Correspondences establishing == The two images are now organized in hierarchy of linked lists in following structure : image=list of columns row=list of words column=list of row word=length (in pixels) At the bottom of the structure, the length of each word is recorded for establishing correspondence between two images to reduce to search only the corresponding structures for the groups of words with the matching lengths. === Seed match finding === A seed match finding is done by comparing each row in image1 with each row in image2. The two rows are then compared to each other by every word. If the length (in pixel) of the two words (one from image1 and one from image2) and their immediate neighbours agree with each other within a predefined tolerance threshold (5 pixels, for example), then they are assumed to match. The row of each image is assumed a match if there are three or more word matches between the two rows. The seed match finding operation is terminated when two pairs of consecutive row match are found. === Match list building === After finishing a seed match finding operation, the next process is to build the match list to generate the correspondences points of the two images. The process is done by searching the matching pairs of rows away from the seed row. == Images mosaicing == Given the list of corresponding points of the two images, finding the transformation of the overlapping portion of the images is the next process. Assuming a pinhole camera model, the transformation between pixels (u,v) of image 1 and pixels (u0, v0) of image 2 is demonstrated by a plane-to-plane projectivity. [ s u ′ s v ′ s ] = [ p 11 p 12 p 13 p 21 p 22 p 23 p 31 p 32 1 ] [ u v 1 ] E q .1 {\displaystyle \left[{\begin{array}{c}su'\\sv'\\s\end{array}}\right]=\left[{\begin{array}{ccc}p_{11}&p_{12}&p_{13}\\p_{21}&p_{22}&p_{23}\\p_{31}&p_{32}&1\end{array}}\right]\left[{\begin{array}{c}u\\v\\1\end{array}}\right]\qquad Eq.1} The parameters of the projectivity is found from four pairs of matching points. RANSAC regression technique is used to reject outlying matches and estimate the projectivity from the remaining good matches. The projectivity is fine-tuned using correlation at the corners of the overlapping portion to obtain four correspondences to sub-pixel accuracy. Therefore, image1 is then transformed into image2's coordinate system using Eq.1. The typical result of the process is shown in Figure 5. === Many images coping === Finally, the whole page composition is built up by mapping all the images into the coordinate system of an "anchor" image, which is normally the one nearest the page center. The transformations to the anchor frame are calculated by concatenating the pair-wise transformations found earlier. The raw document mosaic is shown in Figure 6. However, there might be a problem of non-consecutive images that are overlap. This problem can be solved by performing Hierarchical sub-mosaics. As shown in Figure 7, image1 and image2 are registered, as are image3 and image4, creating two sub-mosaics. These two sub-mosaics are later stitched together in another mosaicing process. == Applied areas == There are various areas that the technique of document mosaicing can be applied to such as : Text segmentation of images of documents Document Recognition Interaction with paper on the digital desk Video mosaics for virtual environments Image registration techniques == Relevant research papers == Huang, T.S.; Netravali, A.N. (1994). "Motion and structure from feature correspondences: A review". Proceedings of the IEEE. 82 (2): 252–268. doi:10.1109/5.265351. D.G. Lowe. [1] Perceptual Organization and Visual Recognition. Kluwer Academic Publishers, Boston, 1985. Irani, M.; Peleg, S. (1991). "Improving resolution by image registration". CVGIP: Graphical Models and Image Processing. 53 (3): 231–239. doi:10.1016/1049-9652(91)90045-L. S2CID 4834546. Shivakumara, P.; Kumar, G. Hemantha; Guru, D. S.; Nagabhushan, P. (2006). "
Terminator (franchise)
Terminator is an American media franchise created by James Cameron and Gale Anne Hurd. It is considered to be of the cyberpunk subgenre of science fiction. The franchise primarily focuses on the events leading to a future post-apocalyptic war between a synthetic intelligence known as Skynet, and a surviving resistance of humans led by John Connor. In this future, Skynet uses an arsenal of cyborgs known as Terminators, designed to mimic humans and infiltrate the resistance. Much of the franchise takes place in time periods prior to the Skynet takeover, with both humans and Terminators using time travel to attempt to alter the past and change the outcome of the future. A prominent Terminator model throughout the films is the T-800, commonly known as "the Terminator", with instances of this model portrayed by Arnold Schwarzenegger. The franchise began with the 1984 film The Terminator, written and directed by Cameron, with Hurd as producer. They would return for the 1991 sequel Terminator 2: Judgment Day (or T2). Both films were critical and commercial successes. Terminator 3: Rise of the Machines (or T3) was released in 2003 to positive reviews, followed by Terminator Salvation in 2009 to more negative reviews. Salvation was intended as the first in a new trilogy, which was later scrapped after the film rights were sold. Cameron was consulted for the 2015 film Terminator Genisys, a reboot branching off from the timeline of the original film. It was negatively received and performed poorly at the box-office. Cameron had a larger role as a producer of the 2019 film Terminator: Dark Fate, a direct sequel to T2 that ignores the three preceding films. As with Salvation, both Genisys and Dark Fate were planned as first installments of new trilogies, with the plans scrapped each time due to the films' poor box-office performances. Outside of the theatrical films, Cameron co-directed T2-3D: Battle Across Time, a 1996 theme park film-based attraction. It was produced as the original sequel to T2 and reunited its main cast. A television series, Terminator: The Sarah Connor Chronicles, was developed without Cameron's involvement and aired for two seasons in 2008 and 2009. It was also produced as a T2 sequel, taking place in an alternate timeline that ignores the third film and subsequent events. Terminator Zero, an anime series, premiered in August 2024. The franchise has also inspired several lines of comic books since 1988, and numerous video games since 1991. By 2010, the franchise had generated $3 billion in revenue. == Themes and setting == The central theme of the franchise is the battle for survival between the nearly-extinct human race and the world-spanning, synthetic intelligence that is Skynet. Skynet is positioned in the first film, The Terminator (1984), as a U.S. strategic "Global Digital Defense Network" computer system by Cyberdyne Systems which becomes self-aware. Shortly after activation, Skynet seemingly perceives all humans as a threat to its existence and formulates a plan to systematically wipe out humanity itself. The system initiates a nuclear first strike against Russia, thereby ensuring a devastating second strike and a nuclear holocaust which wipes out much of humanity in the resulting nuclear war. In the post-apocalyptic aftermath, Skynet later builds up its own autonomous machine-based military capability which includes the Terminators used against individual human targets and thereafter proceeds to wage a persistent total war against the surviving elements of humanity, some of whom have militarily organized themselves into a Resistance. At some point in this future, Skynet develops the capability of time travel and both it and the Resistance seek to use this technology in order to win the war; either by altering or accelerating past events or by preventing the apocalyptic timeline. === Judgment Day === In the franchise, Judgment Day (a reference to the biblical Day of Judgment) is the date on which Skynet becomes self-aware, in which case its creators panic and attempt to deactivate the network. As a result, Skynet perceives humanity as a threat and attempts to exterminate them. Skynet launches an all-out nuclear attack on Russia in order to provoke a nuclear counter-strike against the United States, knowing this will eliminate its human enemies. Due to time travel and the consequent ability to change the future, several differing dates are given for Judgment Day. In Terminator 2: Judgment Day (1991), Sarah Connor states that Judgment Day will occur on August 29, 1997. However, this date is delayed following the attack on Cyberdyne Systems in the same film. Judgment Day has various different dates in different timelines of the subsequent films, as well as the television series, creating a multiverse of temporal phenomena. In Terminator 3: Rise of the Machines (2003) and Terminator Salvation (2009), Judgment Day was postponed to July 2003. In Terminator: The Sarah Connor Chronicles (2008–2009), the attack on Cyberdyne Systems in the second film delayed Judgment Day to April 21, 2011. In Terminator Genisys (2015), the fifth film in the franchise, Judgment Day was postponed to an unspecified day in October 2017, attributed to altered events in both the future and the past. Sarah and Kyle Reese travel through time to the year 2017 and seemingly defeat Skynet, but the system core, contained inside a subterranean blast shelter, survives unknown to them, thus further delaying, rather than preventing, Judgment Day. In Terminator: Dark Fate (2019), the direct sequel to Terminator 2: Judgment Day, a date is not given for the new Judgment Day though it is named as such by Grace. Since Grace is a ten-year-old in 2020 and shown as a teenager in the post-Judgment Day world in flash-forwards throughout the film, Judgment Day occurs sometime in the early 2020s in this timeline. == Franchise rights == Before the first film was created, director James Cameron sold the rights for $1 to Gale Anne Hurd, his future wife, who produced the film, under the strict provision that he be allowed to direct it. Hemdale Film Corporation also became a 50-percent owner of the franchise rights, until its share was sold in 1990 to Carolco Pictures, a company founded by Andrew G. Vajna and Mario Kassar. Terminator 2: Judgment Day was released a year later. Carolco filed for bankruptcy in 1995 and its library was subsequently acquired by StudioCanal, which continues to own the franchise today. However, the rights to future Terminator films were ultimately put up for auction. By that time, Cameron had become interested in making a Terminator 3 film. The rights were ultimately auctioned to Vajna in 1997, for $8 million. Vajna and Kassar spent another $8 million to purchase Hurd's half of the rights in 1998, becoming the full owners of the franchise. Hurd was initially opposed to the sale of the rights, while Cameron had lost interest in the franchise and a third film. After the 2003 release of Terminator 3: Rise of the Machines, the franchise rights were sold in 2007 for about $25 million to The Halcyon Company, which produced Terminator Salvation in 2009. Later that year, the company faced legal issues and filed for bankruptcy, putting the franchise rights up for sale. The rights were valued at about $70 million. In 2010, the rights were sold for $29.5 million to Pacificor, a hedge fund that was Halcyon's largest creditor. In 2012, the rights were sold to Megan Ellison and her production company Annapurna Pictures for less than $20 million, a lower price than what was previously offered. The low price was because of the possibility of Cameron regaining the rights in 2019, as a result of new North American copyright laws. Megan's brother David Ellison and Skydance Productions produced Terminator Genisys in 2015. Cameron worked together with David Ellison to produce the 2019 film Terminator: Dark Fate. As the film neared its release, Hurd filed to terminate a copyright grant made 35 years earlier. Under this move, Hurd would again become a 50-percent owner of the rights with Cameron and Skydance could lose the rights to make any additional Terminator films beginning in November 2020, unless a new deal is worked out. Skydance responded that it had a deal in place with Cameron and that it "controls the rights to the Terminator franchise for the foreseeable future". == Films == === The Terminator (1984) === The Terminator is a 1984 science fiction action film released by Orion Pictures, co-written and directed by James Cameron and starring Arnold Schwarzenegger, Linda Hamilton and Michael Biehn. It is the first work in the Terminator franchise. In the film, robots take over the world in the near future, directed by the artificial intelligence Skynet. With its sole mission to completely annihilate humanity, it develops android assassins called Terminators that outwardly appear human. A man named John Connor starts the Tech-Com resistance to fight the machi
Multi Autonomous Ground-robotic International Challenge
The Multi Autonomous Ground-robotic International Challenge (MAGIC) is a 1.6 million dollar prize competition for autonomous mobile robots funded by TARDEC and the DSTO, the primary research organizations for Tank and Defense research in the United States and Australia respectively. The goal of the competition is to create multi-vehicle robotic teams that can execute an intelligence, surveillance and reconnaissance mission in a dynamic urban environment. The challenge required competitors to map a 500 m x 500 m challenge area in under 3.5 hours and to correctly locate, classify and recognise all simulated threats. The challenge event was conducted in Adelaide, Australia, during November 2010. == Competitors == Initially 12 teams were selected for the competition in November 2009, of which 10 teams received funding. These included: MAGICian – Adelaide/Perth, Australia (UWA, ECU, Flinders, Thales) Strategic Engineering – Adelaide, Australia (U. Adelaide) Northern Hunters – Canada (Royal Military College of Canada) Chiba Team – Japan (Chiba University) Cappadocia – Ankara, Turkey (ASELSAN, Ohio State University) RASR – Gaithersburg, Md. (Robotics Research, LLC; QinetiQ; Embry-Riddle Aeronautical University) Team Cornell – US (Cornell University) Team Michigan – Ann Arbor, Mich. (University of Michigan) Virginia Tech – US (Virginia Tech) University of Pennsylvania – Philadelphia (University of Pennsylvania) Numinence – Brisbane, Australia (Numinence Pty Ltd, La Trobe University) UNSW – Sydney, Australia (UNSW) The first downselection trial required teams to map an indoor area and outdoor area, and to demonstrate distributing and handing over tasks between robots. During the first downselection trial, the top six teams were selected: Cappadocia – Ankara, Turkey MAGICian – Adelaide/Perth, Australia RASR – Gaithersburg, Md. Team Michigan – Ann Arbor, Mich. University of Pennsylvania – Philadelphia Chiba Team – Japan Before the finals were held, Chiba Team withdrew from the competition, leaving five competitors. == Event == Ultimately the overall goal of fully autonomous operations without human intervention was not achieved, however, the Secretary for Defence stated "The competing vehicles demonstrated new advances in robotics technology, which are very promising for their potential deployment in combat zones where they can replace our troops in carrying out life-threatening tasks" and considered the competition a success. == Results == The official results of the competition were: First – Team Michigan ($750,000 prize) Second – University of Pennsylvania ($250,000 prize) Third – RASR ($100,000 prize) Fourth – MAGICian & Cappadocia The "Old Ram Shed Challenge" was a single-day competition held after the completion of MAGIC. It was smaller in scale, allowing all of the teams to demonstrate their systems during a single day. The University of Pennsylvania won this challenge, having found a greater number of the target objects than the other teams. == Technology == Key technology used by all teams was computer vision, sensor fusion, human-robot interaction, and simultaneous localization and mapping (SLAM). Team Michigan, a collaboration between the University of Michigan's APRIL Lab and Soar Technology, Inc., had the largest fleet of 14 robots, developed their own Inertial Measurement Unit, and created their skid steer robot chassis out of Baltic birch plywood. Additionally, they had minimal reliance on GPS and used bandwidth limited 900 MHz radios for all telemetry, imaging, and status communications between all robots and the ground station. The code was written primarily in Java and each robot was equipped with an actuated 2D LIDAR, along with a unique 2D barcode for inter-robot recognition. The University of Pennsylvania team consisted of only four members. All code was written using Matlab. The robots were equipped with omnidirectional vision. RASR used the Foster-Miller TALON vehicle. MAGICian used the WAMbot robots developed by The University of Western Australia, Edith Cowan University and Thales Australia. Code was written in C++ and Java. The robots were equipped with SICK laser scanners. See the September/October 2012 special issue of the Journal of Field Robotics for contest highlights, technical approaches taken by several of the teams, and an explanation of the evaluation metrics used by organizers.
Vehicle infrastructure integration
The Vehicle Infrastructure Integration (VII), also known as "Connected Roadways" or "vehicle-to-everything" (V2X) technology, is a United States Department of Transportation initiative that aims to improve road safety by developing technology that connects road vehicles with their environment. This development draws on several disciplines, including transport engineering, electrical engineering, automotive engineering, telematics, and computer science. Although VII specifically covers road transport, similar technologies are under development for other modes of transport. For example, airplanes may use ground-based beacons for automated guidance, allowing the autopilot to fly the plane without human intervention. == Goals == The goal of VII is to establish a communication link between vehicles (via On-Board Equipment, or OBE) and roadside infrastructure (via Roadside Equipment, or RSE) to enhance the safety, efficiency, and convenience of transportation systems. Two potential approaches are the widespread deployment of a dedicated short-range communications (DSRC) link on the 5.9GHz band, and cellular communication (C-V2X). Either of these methods would allow vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communication. The initiative has three priorities: Stakeholder evaluation and acceptance of the business model and its deployment schedule, Validation of the technology, with a focus on communications systems, in relation to deployment costs, and Creation of legal structures and policies, especially concerning digital privacy, to improve the system's long-term potential for success. === Safety === Current automotive safety technology relies primarily on vehicle-based radar, lidar, and sonar systems. This technology allows, for instance, a potential reduction in rear-end collisions by monitoring obstacles in front of or behind the vehicle and automatically applying the brakes when necessary. This technology, however, is limited by the sensing range of vehicle-based radar, particularly in angled and left-turn collisions, such as a motorist losing control of the vehicle during an impending head-on collision. The rear-end collisions addressed by current technology are generally less severe than angled, left-turn, or head-on collisions. VII promotes the development of a direct communication link between road vehicles and all other vehicles nearby, allowing for the exchange of information on vehicle speed and orientation or driver awareness and intent. This real-time exchange of information may enable more effective automated emergency maneuvers, such as steering, decelerating, or braking. In addition to nearby vehicle awareness, VII promotes a communication link between vehicles and roadway infrastructure. Such a link may allow for improved real-time traffic information, better queue management, and feedback to vehicles. Existing implementations of VII use vehicle-based sensors that can recognize and respond to roadway markings or signs, automatically adjusting vehicle parameters to follow the recognized instructions. However, this information may also be acquired via roadside beacons or stored in a centralized database accessible to all vehicles. === Efficiency === With a VII system in place, vehicles will be linked together. The headway between vehicles may therefore be reduced so that there is less empty space on the road, increasing the available capacity per lane. More capacity per lane will in turn imply fewer lanes in general, possibly satisfying the community's concerns about the impact of roadway widening. VII will enable precise traffic-signal coordination by tracking vehicle platoons and will benefit from accurate timing by drawing on real-time traffic data covering volume, density, and turning movements. Real-time traffic data can also be used in the design of new roadways or modification of existing systems as the data could be used to provide accurate origin-destination studies and turning-movement counts for uses in transportation forecasting and traffic operations. Such technology would also lead to improvements for transport engineers to address problems whilst reducing the cost of obtaining and compiling data. Tolling is another prospect for VII technology as it could enable roadways to be automatically tolled. Data could be collectively transmitted to road users for in-vehicle display, outlining the lowest cost, shortest distance, and/or fastest route to a destination on the basis of real-time conditions. === Existing applications === To some extent, results along these lines have been achieved in trials performed around the globe, making use of GPS, mobile phone signals, and vehicle registration plates. GPS is becoming standard in many new high-end vehicles and is an option on most new low- and mid-range vehicles. In addition, many users also have mobile phones that transmit trackable signals (and may also be GPS-enabled). Mobile phones can already be traced for purposes of emergency response. GPS and mobile phone tracking, however, do not provide fully reliable data. Furthermore, integrating mobile phones in vehicles may be prohibitively difficult. Data from mobile phones, though useful, might even increase risks to motorists as they tend to look at their phones rather than concentrate on their driving. Automatic registration plate recognition can provide large quantities of data, but continuously tracking a vehicle through a corridor is a difficult task with existing technology. Today's equipment is designed for data acquisition and functions such as enforcement and tolling, not for returning data to vehicles or motorists for response. GPS will nevertheless be one of the key components in VII systems. == Limitations == === Privacy === VII architecture is designed to prevent identification of individual vehicles, with all data exchange between the vehicle and the system occurring anonymously. Exchanges between the vehicles and third parties such as OEMs and toll collectors will occur, but the network traffic will be sent via encrypted tunnels and will therefore not be decipherable by the VII system. Data sharing with law enforcement or Homeland Security was not included in system design as of 2006. === Technical issues === ==== Coordination ==== A major issue facing the deployment of VII is the problem of how to set up the system initially. The costs associated with installing the technology in vehicles and providing communications and power at every intersection are significant. ==== Maintenance ==== Another factor for consideration in regard to the technology's distribution is how to update and maintain the units. Traffic systems are highly dynamic, with new traffic controls implemented every day and roadways constructed or repaired every year. The vehicle-based option could be updated via the internet (preferably wireless) but may subsequently require all users to have access to internet technology. Alternatively, if receivers were placed in all vehicles and the VII system was primarily located along the roadside, information could be stored in a centralized database. This would allow the agency responsible to issue updates at any time. These would then be disseminated to the roadside units for passing motorists. Operationally, this method is currently considered to provide the greatest effectiveness but at a high cost to the authorities. ==== Security ==== Security of the units is another concern, especially in light of the public acceptance issue. Criminals could tamper, remove, or destroy VII units regardless of whether they are installed inside vehicles or along the roadside. Magnets, electric shocks, and malicious software (viruses, hacking, or jamming) could be used to damage VII systems – regardless of whether units are located inside vehicle or along the roadside. == Recent developments == Much of the current research and experimentation is conducted in the United States where coordination is ensured through the Vehicle Infrastructure Integration Consortium; consisting of automobile manufacturers (Ford, General Motors, Daimler Chrysler, Toyota, Nissan, Honda, Volkswagen, BMW), IT suppliers, U.S. Federal and state transportation departments, and professional associations. Trialing is taking place in Michigan and California. The specific applications now being developed under the U.S. initiative are: Warning drivers of unsafe conditions or imminent collisions. Warning drivers if they are about to run off the road or speed around a curve too fast. Informing system operators of real-time congestion, weather conditions and incidents. Providing operators with information on corridor capacity for real-time management, planning and provision of corridor-wide advisories to drivers. In mid-2007, a VII environment covering some 20 square miles (52 km2) near Detroit was used to test 20 prototype VII applications. Several automobile manufacturers are also conducting their own VII research and triali
Altibase
Altibase is a hybrid database, relational database management system manufactured by the Altibase Corporation. The software's hybrid architecture allows it to access both memory-resident and disk-resident tables using single interface. It supports both synchronous and asynchronous replication and offers real-time ACID compliance. Support is also offered for a variety of SQL standards and programming languages. Other important capabilities include data import and export, data encryption for security, multiple data access command sets, materialized view and temporary tables, and others. == History == From 1991 through 1997 the Mr. RT project was an in-memory database research project, conducted by the Electronics and Telecommunications Research Institute a government-funded research organization in South Korea. Altibase was incorporated in 1999. Altibase acquired an in-memory database engine from the Electronics and Telecommunications Research Institute in February 2000, and commercialized the database in October of the same year. In 2001, Altibase changed the name of the in-memory database product from "Spiner" to "Altibase" in 2001. In 2004, Altibase integrated the in-memory database with a disk-resident database to create a hybrid DBMS, released version 4.0 and renamed it as ALTIBASE HDB. Altibase released version 5.5.1 and 6.1.1 in 2012, version 6.3.1 in November 2013, and 6.5.1 in May 2015. Altibase claims that this is the world's first hybrid DBMS. Altibase released its open source edition version 7.1, however, closed the source in 2023. In August 2023, Altibase released its cloud-optimized version 7.3. === Awards === In 2006, Received the Presidential Award at the Korea Software Awards In 2007, Selected as World-Class Product by the Ministry of Commerce, Industry and Energy In 2009, Awarded the Outstanding Product Award in China's Telecommunications Industry In 2009, Received Outstanding Product Award at the China Billing China 2009 Telecommunication Industry Awards In 2010, Commendation from the Minister of Knowledge Economy for Technological Practicalization In 2011, Received the Grand Prize at the 10th Software Enterprise Competitiveness Award In 2011, Selected as Top 10 Emerging Technologies and received Special Award at the Korea Technology Grand Prize In 2012, Awarded for Contributions to Military Manpower Administration In 2014~2016, Included in Gartner Magic Quadrant for Operational DBMS In 2015, Selected as Outstanding BSS by China Fujian Mobile. In 2023, Awarded as the Excellent Research and Development Institution by the Korean Ministry Science and ICT In 2023, Won the Global Premium Commercial Software Presidential Award at the 9th Global Commercial Software Grand Exhibition in Korea === Release === The first version, called Spiner, was released in 2000 for commercial use. It took half of the in-memory DBMS market share in South Korea. In 2002 the second version was released renamed to Altibase v2.0. By 2003, Altibase v3.0 was released and it entered the Chinese market. Released version 4.0 with hybrid architecture, combining RAM and disk databases, was released in 2004. In 2005 Altibase began working with Chinese telecommunications providers for billing systems, and some financial companies in Taiwan, China, for home trading systems. The software was certified by the Telecommunications Technology Association. The Ministry of Government Administration and Home Affairs gave it an award in 2006. Offices in China and United States opened in 2009. In 2011, version 5.5.1 was renamed it to HDB (for "hybrid database"). The Altibase Data Stream product for complex event processing was renamed DSM. The product received a Korean technology award. Altibase introduced certification services. In 2012, HDB Zeta and Extreme were announced, and DSM renamed to CEP. In 2013, yet another variant called XDB was announced, and the company received ISO/IEC 20000 certification. In 2018, Altibase went open source. Altibase went open source in February, 2018. Altibase Corp has made the decision to discontinue the Altibase 7.1 open source edition, effective March 17, 2023. As a result, the open-source edition of Altibase 7.1 will no longer be available for download or use. Altibase released version 7.3 in September, 2023, its notable feature is the world’s first hybrid partition, allowing data to be stored in both memory and on disk at the partition level. Version 7.3 also added parallel processing capabilities for high-speed performance in both partitioned and non-partitioned scenarios. Improving potential bottlenecks associated with Commit and logging that impact transaction performance, version 7.3 has achieved an approximately 490% enhancement in performance compared to previous versions. === Release history === == Clients == According to marketing research, Altibase have over 700 customers and more than 8,000 of installations and deployments, including 22 Fortune Global 500 Companies. Altibase's clients in the telecommunications, financial services, manufacturing, and utilities sectors include Bloomberg, AT&T, LG, Intel, LGU+, ETRADE, HP, UAT Inc., POSCO, SK Telecom, KT Corporation, Samsung Electronics, Shinhan Bank, Woori Bank, Canon(Toshiba), Hanhwa, The South Korean Ministry of Defense, G-Market, CJ, and Chung-Ang University. === Global clients === Japan FX Prime, a foreign exchange services company Retela Crea Securities United States AT&T Implemented Altibase for its PS-LTE Safety network, where the Presence service plays a vital role. This service handles the reception and storage of user information, conducting real-time checks for online presence and location as needed. Canada Telus One of the major telecommunication companies. Utilizes Altibase for its operations involving real-time user management, processing high volumes of dedicated terminal data, and managing real-time location information (GIS) for terminals. Altibase contributes to the company's in-house solution for maintaining uninterrupted services during national disasters or similar situations, ensuring efficiency and reliability. China China Mobile, China Unicom, China Telecom The three major telecommunications companies. Utilize ALTIBASE HDB in 29 of 31 Chinese provinces. Turkish Ziraat Bank, Halk Bank, Deniz Bank, Garanti BBVA, TEB, Oyak Bank, QNB, Burgan Bank, and others. In 2018, Altibase entered the market through a partnership with ATP-Tradesoft, a subsidiary of Ata Holdings. Collaborating with ATP-Tradesoft. Altibase integrated into the Online Trading System XFront. This integration was well-received by major financial institutions and securities firms in Turkey. Altibase is currently implemented in the XFront Online Trading System, used by 13 significant financial institutions and banks in the Turkey. Thailand Bualuang Securities Altibase has been supplied its DBMS to support the construction of the online stock trading platform. Mongolia MobiCom The Mongolian telecommunication giant, has adopted Altibase’s 7.0 version for its mobile platform for storing the infrequently used data. Azerbaijan M1 highway Altibase has been supplied as the Database Management System (DBMS) for the electronic toll collection system. One of the most crucial transportation networks in the country. India State-owned Karur Vysya Bank In 2013, Altibase provided its hybrid database solution and was deployed for the online banking system === Industries === Telecommunications LGU+ SK Telecom KT Corporation AT&T Telus Financial services Shinhan Bank Woori Bank KakaoPay Securities Implemented Altibase in its stock trading system Leveraging Altibase's replication feature, along with offline replication through shared disk and adapter functionality, the system ensures a high level of availability and consistency, with a reliability rate of 99.999% even in the event of system failures. COREDAX Cryptocurrency market Altibase has entered into a strategic partnership by signing a database management system (DBMS) supply contract with the cryptocurrency exchange Bloomberg ETRADE Manufacturing Samsung Electronics LG POSCO Hanhwa Canon(Toshiba) Intel HP Utilities South Korean Ministry of Defense G-Market CJ UAT Inc. Chung-Ang University == Features == Altibase is a so-called "hybrid DBMS", meaning that it simultaneously supports access to both memory-resident and disk-resident tables via a single interface. It is compatible with Solaris, HP-UX, AIX, Linux, and Windows. It supports the complete SQL standard, features Multiversion concurrency control (MVCC), implements Fuzzy and Ping-Pong Checkpointing for periodically backing up memory-resident data, and ships with Replication and Database Link functionality. High performance, large -capacity service Fast real-time data processing and large amounts of data stable Provide parallel processing architecture for large data management Developed and provided Hybrid Partitioned Table function for efficiency according to data personality High stability
Thompson sampling
Thompson sampling, named after William R. Thompson, is a heuristic for choosing actions that address the exploration–exploitation dilemma in the multi-armed bandit problem. It consists of choosing the action that maximizes the expected reward with respect to a randomly drawn belief. == Description == Consider a set of contexts X {\displaystyle {\mathcal {X}}} , a set of actions A {\displaystyle {\mathcal {A}}} , and rewards in R {\displaystyle \mathbb {R} } . The aim of the player is to play actions under the various contexts, such as to maximize the cumulative rewards. Specifically, in each round, the player obtains a context x ∈ X {\displaystyle x\in {\mathcal {X}}} , plays an action a ∈ A {\displaystyle a\in {\mathcal {A}}} and receives a reward r ∈ R {\displaystyle r\in \mathbb {R} } following a distribution that depends on the context and the issued action. The elements of Thompson sampling are as follows: a likelihood function P ( r | θ , a , x ) {\displaystyle P(r|\theta ,a,x)} ; a set Θ {\displaystyle \Theta } of parameters θ {\displaystyle \theta } of the distribution of r {\displaystyle r} ; a prior distribution P ( θ ) {\displaystyle P(\theta )} on these parameters; past observations triplets D = { ( x ; a ; r ) } {\displaystyle {\mathcal {D}}=\{(x;a;r)\}} ; a posterior distribution P ( θ | D ) ∝ P ( D | θ ) P ( θ ) {\displaystyle P(\theta |{\mathcal {D}})\propto P({\mathcal {D}}|\theta )P(\theta )} , where P ( D | θ ) {\displaystyle P({\mathcal {D}}|\theta )} is the likelihood function. Thompson sampling consists of playing the action a ∗ ∈ A {\displaystyle a^{\ast }\in {\mathcal {A}}} according to the probability that it maximizes the expected reward; action a ∗ {\displaystyle a^{\ast }} is chosen with probability ∫ I [ E ( r | a ∗ , x , θ ) = max a ′ E ( r | a ′ , x , θ ) ] P ( θ | D ) d θ , {\displaystyle \int \mathbb {I} \left[\mathbb {E} (r|a^{\ast },x,\theta )=\max _{a'}\mathbb {E} (r|a',x,\theta )\right]P(\theta |{\mathcal {D}})d\theta ,} where I {\displaystyle \mathbb {I} } is the indicator function. In practice, the rule is implemented by sampling. In each round, parameters θ ∗ {\displaystyle \theta ^{\ast }} are sampled from the posterior P ( θ | D ) {\displaystyle P(\theta |{\mathcal {D}})} , and an action a ∗ {\displaystyle a^{\ast }} chosen that maximizes E [ r | θ ∗ , a ∗ , x ] {\displaystyle \mathbb {E} [r|\theta ^{\ast },a^{\ast },x]} , i.e. the expected reward given the sampled parameters, the action, and the current context. Conceptually, this means that the player instantiates their beliefs randomly in each round according to the posterior distribution, and then acts optimally according to them. In most practical applications, it is computationally onerous to maintain and sample from a posterior distribution over models. As such, Thompson sampling is often used in conjunction with approximate sampling techniques. == History == Thompson sampling was originally described by Thompson in 1933. It was subsequently rediscovered numerous times independently in the context of multi-armed bandit problems. A first proof of convergence for the bandit case has been shown in 1997. The first application to Markov decision processes was in 2000. A related approach (see Bayesian control rule) was published in 2010. In 2010 it was also shown that Thompson sampling is instantaneously self-correcting. Asymptotic convergence results for contextual bandits were published in 2011. Thompson Sampling has been widely used in many online learning problems including A/B testing in website design and online advertising, and accelerated learning in decentralized decision making. A Double Thompson Sampling (D-TS) algorithm has been proposed for dueling bandits, a variant of traditional MAB, where feedback comes in the form of pairwise comparison. == Relationship to other approaches == === Probability matching === Probability matching is a decision strategy in which predictions of class membership are proportional to the class base rates. Thus, if in the training set positive examples are observed 60% of the time, and negative examples are observed 40% of the time, the observer using a probability-matching strategy will predict (for unlabeled examples) a class label of "positive" on 60% of instances, and a class label of "negative" on 40% of instances. === Bayesian control rule === A generalization of Thompson sampling to arbitrary dynamical environments and causal structures, known as Bayesian control rule, has been shown to be the optimal solution to the adaptive coding problem with actions and observations. In this formulation, an agent is conceptualized as a mixture over a set of behaviours. As the agent interacts with its environment, it learns the causal properties and adopts the behaviour that minimizes the relative entropy to the behaviour with the best prediction of the environment's behaviour. If these behaviours have been chosen according to the maximum expected utility principle, then the asymptotic behaviour of the Bayesian control rule matches the asymptotic behaviour of the perfectly rational agent. The setup is as follows. Let a 1 , a 2 , … , a T {\displaystyle a_{1},a_{2},\ldots ,a_{T}} be the actions issued by an agent up to time T {\displaystyle T} , and let o 1 , o 2 , … , o T {\displaystyle o_{1},o_{2},\ldots ,o_{T}} be the observations gathered by the agent up to time T {\displaystyle T} . Then, the agent issues the action a T + 1 {\displaystyle a_{T+1}} with probability: P ( a T + 1 | a ^ 1 : T , o 1 : T ) , {\displaystyle P(a_{T+1}|{\hat {a}}_{1:T},o_{1:T}),} where the "hat"-notation a ^ t {\displaystyle {\hat {a}}_{t}} denotes the fact that a t {\displaystyle a_{t}} is a causal intervention (see Causality), and not an ordinary observation. If the agent holds beliefs θ ∈ Θ {\displaystyle \theta \in \Theta } over its behaviors, then the Bayesian control rule becomes P ( a T + 1 | a ^ 1 : T , o 1 : T ) = ∫ Θ P ( a T + 1 | θ , a ^ 1 : T , o 1 : T ) P ( θ | a ^ 1 : T , o 1 : T ) d θ {\displaystyle P(a_{T+1}|{\hat {a}}_{1:T},o_{1:T})=\int _{\Theta }P(a_{T+1}|\theta ,{\hat {a}}_{1:T},o_{1:T})P(\theta |{\hat {a}}_{1:T},o_{1:T})\,d\theta } , where P ( θ | a ^ 1 : T , o 1 : T ) {\displaystyle P(\theta |{\hat {a}}_{1:T},o_{1:T})} is the posterior distribution over the parameter θ {\displaystyle \theta } given actions a 1 : T {\displaystyle a_{1:T}} and observations o 1 : T {\displaystyle o_{1:T}} . In practice, the Bayesian control amounts to sampling, at each time step, a parameter θ ∗ {\displaystyle \theta ^{\ast }} from the posterior distribution P ( θ | a ^ 1 : T , o 1 : T ) {\displaystyle P(\theta |{\hat {a}}_{1:T},o_{1:T})} , where the posterior distribution is computed using Bayes' rule by only considering the (causal) likelihoods of the observations o 1 , o 2 , … , o T {\displaystyle o_{1},o_{2},\ldots ,o_{T}} and ignoring the (causal) likelihoods of the actions a 1 , a 2 , … , a T {\displaystyle a_{1},a_{2},\ldots ,a_{T}} , and then by sampling the action a T + 1 ∗ {\displaystyle a_{T+1}^{\ast }} from the action distribution P ( a T + 1 | θ ∗ , a ^ 1 : T , o 1 : T ) {\displaystyle P(a_{T+1}|\theta ^{\ast },{\hat {a}}_{1:T},o_{1:T})} . === Upper-confidence-bound (UCB) algorithms === Thompson sampling and upper-confidence bound algorithms share a fundamental property that underlies many of their theoretical guarantees. Roughly speaking, both algorithms allocate exploratory effort to actions that might be optimal and are in this sense "optimistic". Leveraging this property, one can translate regret bounds established for UCB algorithms to Bayesian regret bounds for Thompson sampling or unify regret analysis across both these algorithms and many classes of problems.