Change data capture

Change data capture

In databases, change data capture (CDC) is a set of software design patterns used to determine and track the data that has changed (the "deltas") so that action can be taken using the changed data. The result is a delta-driven dataset. CDC is an approach to data integration that is based on the identification, capture and delivery of the changes made to enterprise data sources. For instance it can be used for incremental update of data loading. CDC occurs often in data warehouse environments since capturing and preserving the state of data across time is one of the core functions of a data warehouse, but CDC can be utilized in any database or data repository system. == Methodology == System developers can set up CDC mechanisms in a number of ways and in any one or a combination of system layers from application logic down to physical storage. In a simplified CDC context, one computer system has data believed to have changed from a previous point in time, and a second computer system needs to take action based on that changed data. The former is the source, the latter is the target. It is possible that the source and target are the same system physically, but that would not change the design pattern logically. Multiple CDC solutions can exist in a single system. === Timestamps on rows === Tables whose changes must be captured may have a column that represents the time of last change. Names such as LAST_UPDATE, LAST_MODIFIED, etc. are common. Any row in any table that has a timestamp in that column that is more recent than the last time data was captured is considered to have changed. Timestamps on rows are also frequently used for optimistic locking so this column is often available. === Version numbers on rows === Database designers give tables whose changes must be captured a column that contains a version number. Names such as VERSION_NUMBER, etc. are common. One technique is to mark each changed row with a version number. A current version is maintained for the table, or possibly a group of tables. This is stored in a supporting construct such as a reference table. When a change capture occurs, all data with the latest version number is considered to have changed. Once the change capture is complete, the reference table is updated with a new version number. (Do not confuse this technique with row-level versioning used for optimistic locking. For optimistic locking each row has an independent version number, typically a sequential counter. This allows a process to atomically update a row and increment its counter only if another process has not incremented the counter. But CDC cannot use row-level versions to find all changes unless it knows the original "starting" version of every row. This is impractical to maintain.) === Status indicators on rows === This technique can either supplement or complement timestamps and versioning. It can configure an alternative if, for example, a status column is set up on a table row indicating that the row has changed (e.g., a boolean column that, when set to true, indicates that the row has changed). Otherwise, it can act as a complement to the previous methods, indicating that a row, despite having a new version number or a later date, still shouldn't be updated on the target (for example, the data may require human validation). === Time/version/status on rows === This approach combines the three previously discussed methods. As noted, it is not uncommon to see multiple CDC solutions at work in a single system, however, the combination of time, version, and status provides a particularly powerful mechanism and programmers should utilize them as a trio where possible. The three elements are not redundant or superfluous. Using them together allows for such logic as, "Capture all data for version 2.1 that changed between 2005-06-01 00:00 and 2005-07-01 00:00 where the status code indicates it is ready for production." === Triggers on tables === May include a publish/subscribe pattern to communicate the changed data to multiple targets. In this approach, triggers log events that happen to the transactional table into another queue table that can later be "played back". For example, imagine an Accounts table, when transactions are taken against this table, triggers would fire that would then store a history of the event or even the deltas into a separate queue table. The queue table might have schema with the following fields: Id, TableName, RowId, Timestamp, Operation. The data inserted for our Account sample might be: 1, Accounts, 76, 2008-11-02 00:15, Update. More complicated designs might log the actual data that changed. This queue table could then be "played back" to replicate the data from the source system to a target. Data capture offers a challenge in that the structure, contents and use of a transaction log is specific to a database management system. Unlike data access, no standard exists for transaction logs. Most database management systems do not document the internal format of their transaction logs, although some provide programmatic interfaces to their transaction logs (for example: Oracle, DB2, SQL/MP, SQL/MX and SQL Server 2008). Other challenges in using transaction logs for change data capture include: Coordinating the reading of the transaction logs and the archiving of log files (database management software typically archives log files off-line on a regular basis). Translation between physical storage formats that are recorded in the transaction logs and the logical formats typically expected by database users (e.g., some transaction logs save only minimal buffer differences that are not directly useful for change consumers). Dealing with changes to the format of the transaction logs between versions of the database management system. Eliminating uncommitted changes that the database wrote to the transaction log and later rolled back. Dealing with changes to the metadata of tables in the database. CDC solutions based on transaction log files have distinct advantages that include: minimal impact on the database (even more so if one uses log shipping to process the logs on a dedicated host). no need for programmatic changes to the applications that use the database. low latency in acquiring changes. transactional integrity: log scanning can produce a change stream that replays the original transactions in the order they were committed. Such a change stream include changes made to all tables participating in the captured transaction. no need to change the database schema == Confounding factors == As often occurs in complex domains, the final solution to a CDC problem may have to balance many competing concerns. === Unsuitable source systems === Change data capture both increases in complexity and reduces in value if the source system saves metadata changes when the data itself is not modified. For example, some Data models track the user who last looked at but did not change the data in the same structure as the data. This results in noise in the Change Data Capture. === Tracking the capture === Actually tracking the changes depends on the data source. If the data is being persisted in a modern database then Change Data Capture is a simple matter of permissions. Two techniques are in common use: Tracking changes using database triggers Reading the transaction log as, or shortly after, it is written. If the data is not in a modern database, CDC becomes a programming challenge. === Push versus pull === Push: the source process creates a snapshot of changes within its own process and delivers rows downstream. The downstream process uses the snapshot, creates its own subset and delivers them to the next process. Pull: the target that is immediately downstream from the source, prepares a request for data from the source. The downstream target delivers the snapshot to the next target, as in the push model. === Alternatives === Sometimes the slowly changing dimension is used as an alternative method. CDC and SCD are similar in that both methods can detect changes in a data set. The most common forms of SCD are type 1 (overwrite), type 2 (maintain history) or 3 (only previous and current value). SCD 2 can be useful if history is needed in the target system. CDC overwrites in the target system (akin to SCD1), and is ideal when only the changed data needs to arrive at the target, i.e. a delta-driven dataset.

Anomaly detection

In data analysis, anomaly detection (also referred to as outlier detection and sometimes as novelty detection) is generally understood to be the identification of rare items, events or observations which deviate significantly from the majority of the data and do not conform to a well defined notion of normal behavior. Such examples may arouse suspicions of being generated by a different mechanism, or appear inconsistent with the remainder of that set of data. Anomaly detection finds application in many domains including cybersecurity, medicine, machine vision, statistics, neuroscience, law enforcement and financial fraud to name only a few. Anomalies were initially searched for clear rejection or omission from the data to aid statistical analysis, for example to compute the mean or standard deviation. They were also removed to better predictions from models such as linear regression, and more recently their removal aids the performance of machine learning algorithms. However, in many applications anomalies themselves are of interest and are the observations most desirous in the entire data set, which need to be identified and separated from noise or irrelevant outliers. Three broad categories of anomaly detection techniques exist. Supervised anomaly detection techniques require a data set that has been labeled as "normal" and "abnormal" and involves training a classifier. However, this approach is rarely used in anomaly detection due to the general unavailability of labelled data and the inherent unbalanced nature of the classes. Semi-supervised anomaly detection techniques assume that some portion of the data is labelled. This may be any combination of the normal or anomalous data, but more often than not, the techniques construct a model representing normal behavior from a given normal training data set, and then test the likelihood of a test instance to be generated by the model. Unsupervised anomaly detection techniques assume the data is unlabelled and are by far the most commonly used due to their wider and relevant application. == Definition == Many attempts have been made in the statistical and computer science communities to define an anomaly. The most prevalent ones include the following, and can be categorised into three groups: those that are ambiguous, those that are specific to a method with pre-defined thresholds usually chosen empirically, and those that are formally defined: === Ill defined === An outlier is an observation which deviates so much from the other observations as to arouse suspicions that it was generated by a different mechanism. Anomalies are instances or collections of data that occur very rarely in the data set and whose features differ significantly from most of the data. An outlier is an observation (or subset of observations) which appears to be inconsistent with the remainder of that set of data. An anomaly is a point or collection of points that is relatively distant from other points in multi-dimensional space of features. Anomalies are patterns in data that do not conform to a well-defined notion of normal behaviour. === Specific === Let T be observations from a univariate Gaussian distribution and O a point from T. Then the z-score for O is greater than a pre-selected threshold if and only if O is an outlier. == History == === Intrusion detection === The concept of intrusion detection, a critical component of anomaly detection, has evolved significantly over time. Initially, it was a manual process where system administrators would monitor for unusual activities, such as a vacationing user's account being accessed or unexpected printer activity. This approach was not scalable and was soon superseded by the analysis of audit logs and system logs for signs of malicious behavior. By the late 1970s and early 1980s, the analysis of these logs was primarily used retrospectively to investigate incidents, as the volume of data made it impractical for real-time monitoring. The affordability of digital storage eventually led to audit logs being analyzed online, with specialized programs being developed to sift through the data. These programs, however, were typically run during off-peak hours due to their computational intensity. The 1990s brought the advent of real-time intrusion detection systems capable of analyzing audit data as it was generated, allowing for immediate detection of and response to attacks. This marked a significant shift towards proactive intrusion detection. As the field has continued to develop, the focus has shifted to creating solutions that can be efficiently implemented across large and complex network environments, adapting to the ever-growing variety of security threats and the dynamic nature of modern computing infrastructures. == Applications == Anomaly detection is applicable in a very large number and variety of domains, and is an important subarea of unsupervised machine learning. As such it has applications in cyber-security, intrusion detection, fraud detection, fault detection, system health monitoring, event detection in sensor networks, detecting ecosystem disturbances, defect detection in images using machine vision, medical diagnosis and law enforcement. === Intrusion detection === Anomaly detection was proposed for intrusion detection systems (IDS) by Dorothy Denning in 1986. Anomaly detection for IDS is normally accomplished with thresholds and statistics, but can also be done with soft computing, and inductive learning. Types of features proposed by 1999 included profiles of users, workstations, networks, remote hosts, groups of users, and programs based on frequencies, means, variances, covariances, and standard deviations. The counterpart of anomaly detection in intrusion detection is misuse detection. === Fintech fraud detection === Anomaly detection is vital in fintech for fraud prevention. === Preprocessing === Preprocessing data to remove anomalies can be an important step in data analysis, and is done for a number of reasons. Statistics such as the mean and standard deviation are more accurate after the removal of anomalies, and the visualisation of data can also be improved. In supervised learning, removing the anomalous data from the dataset often results in a statistically significant increase in accuracy. === Video surveillance === Anomaly detection has become increasingly vital in video surveillance to enhance security and safety. With the advent of deep learning technologies, methods using Convolutional Neural Networks (CNNs) and Simple Recurrent Units (SRUs) have shown significant promise in identifying unusual activities or behaviors in video data. These models can process and analyze extensive video feeds in real-time, recognizing patterns that deviate from the norm, which may indicate potential security threats or safety violations. An important aspect for video surveillance is the development of scalable real-time frameworks. Such pipelines are required for processing multiple video streams with low computational resources. === IT infrastructure === In IT infrastructure management, anomaly detection is crucial for ensuring the smooth operation and reliability of services. These are complex systems, composed of many interactive elements and large data quantities, requiring methods to process and reduce this data into a human and machine interpretable format. Techniques like the IT Infrastructure Library (ITIL) and monitoring frameworks are employed to track and manage system performance and user experience. Detected anomalies can help identify and pre-empt potential performance degradations or system failures, thus maintaining productivity and business process effectiveness. === IoT systems === Anomaly detection is critical for the security and efficiency of Internet of Things (IoT) systems. It helps in identifying system failures and security breaches in complex networks of IoT devices. The methods must manage real-time data, diverse device types, and scale effectively. Garg et al. have introduced a multi-stage anomaly detection framework that improves upon traditional methods by incorporating spatial clustering, density-based clustering, and locality-sensitive hashing. This tailored approach is designed to better handle the vast and varied nature of IoT data, thereby enhancing security and operational reliability in smart infrastructure and industrial IoT systems. === Petroleum industry === Anomaly detection is crucial in the petroleum industry for monitoring critical machinery. A 2015 paper proposed a novel segmentation algorithm using support vector machines to analyze sensor data for real-time anomaly detection. === Oil and gas pipeline monitoring === In the oil and gas sector, anomaly detection is not just crucial for maintenance and safety, but also for environmental protection. Aljameel et al. propose an advanced machine learning-based model for detecting minor leaks in oil and gas pipelines, a task traditional methods may miss.

Multi Autonomous Ground-robotic International Challenge

The Multi Autonomous Ground-robotic International Challenge (MAGIC) is a 1.6 million dollar prize competition for autonomous mobile robots funded by TARDEC and the DSTO, the primary research organizations for Tank and Defense research in the United States and Australia respectively. The goal of the competition is to create multi-vehicle robotic teams that can execute an intelligence, surveillance and reconnaissance mission in a dynamic urban environment. The challenge required competitors to map a 500 m x 500 m challenge area in under 3.5 hours and to correctly locate, classify and recognise all simulated threats. The challenge event was conducted in Adelaide, Australia, during November 2010. == Competitors == Initially 12 teams were selected for the competition in November 2009, of which 10 teams received funding. These included: MAGICian – Adelaide/Perth, Australia (UWA, ECU, Flinders, Thales) Strategic Engineering – Adelaide, Australia (U. Adelaide) Northern Hunters – Canada (Royal Military College of Canada) Chiba Team – Japan (Chiba University) Cappadocia – Ankara, Turkey (ASELSAN, Ohio State University) RASR – Gaithersburg, Md. (Robotics Research, LLC; QinetiQ; Embry-Riddle Aeronautical University) Team Cornell – US (Cornell University) Team Michigan – Ann Arbor, Mich. (University of Michigan) Virginia Tech – US (Virginia Tech) University of Pennsylvania – Philadelphia (University of Pennsylvania) Numinence – Brisbane, Australia (Numinence Pty Ltd, La Trobe University) UNSW – Sydney, Australia (UNSW) The first downselection trial required teams to map an indoor area and outdoor area, and to demonstrate distributing and handing over tasks between robots. During the first downselection trial, the top six teams were selected: Cappadocia – Ankara, Turkey MAGICian – Adelaide/Perth, Australia RASR – Gaithersburg, Md. Team Michigan – Ann Arbor, Mich. University of Pennsylvania – Philadelphia Chiba Team – Japan Before the finals were held, Chiba Team withdrew from the competition, leaving five competitors. == Event == Ultimately the overall goal of fully autonomous operations without human intervention was not achieved, however, the Secretary for Defence stated "The competing vehicles demonstrated new advances in robotics technology, which are very promising for their potential deployment in combat zones where they can replace our troops in carrying out life-threatening tasks" and considered the competition a success. == Results == The official results of the competition were: First – Team Michigan ($750,000 prize) Second – University of Pennsylvania ($250,000 prize) Third – RASR ($100,000 prize) Fourth – MAGICian & Cappadocia The "Old Ram Shed Challenge" was a single-day competition held after the completion of MAGIC. It was smaller in scale, allowing all of the teams to demonstrate their systems during a single day. The University of Pennsylvania won this challenge, having found a greater number of the target objects than the other teams. == Technology == Key technology used by all teams was computer vision, sensor fusion, human-robot interaction, and simultaneous localization and mapping (SLAM). Team Michigan, a collaboration between the University of Michigan's APRIL Lab and Soar Technology, Inc., had the largest fleet of 14 robots, developed their own Inertial Measurement Unit, and created their skid steer robot chassis out of Baltic birch plywood. Additionally, they had minimal reliance on GPS and used bandwidth limited 900 MHz radios for all telemetry, imaging, and status communications between all robots and the ground station. The code was written primarily in Java and each robot was equipped with an actuated 2D LIDAR, along with a unique 2D barcode for inter-robot recognition. The University of Pennsylvania team consisted of only four members. All code was written using Matlab. The robots were equipped with omnidirectional vision. RASR used the Foster-Miller TALON vehicle. MAGICian used the WAMbot robots developed by The University of Western Australia, Edith Cowan University and Thales Australia. Code was written in C++ and Java. The robots were equipped with SICK laser scanners. See the September/October 2012 special issue of the Journal of Field Robotics for contest highlights, technical approaches taken by several of the teams, and an explanation of the evaluation metrics used by organizers.

Sword Art Online

Sword Art Online (Japanese: ソードアート・オンライン, Hepburn: Sōdo Āto Onrain) is a Japanese light novel series written by Reki Kawahara and illustrated by abec. The series takes place in the 2020s and focuses on protagonists Kazuto "Kirito" Kirigaya and Asuna Yuuki as they play through various virtual reality MMORPG worlds, and later their involvement in the matters of a simulated civilization. Kawahara originally released the series as a web novel on his website from 2002 to 2008. The light novels began publication on ASCII Media Works' Dengeki Bunko imprint from April 10, 2009, with a spin-off series launching in October 2012. The series has spawned twelve manga adaptations published by ASCII Media Works and Kadokawa. The novels and the manga adaptations have been licensed for release in North America by Yen Press. An anime television series produced by A-1 Pictures, known simply as Sword Art Online, aired in Japan between July and December 2012, with a television film Sword Art Online: Extra Edition airing on December 31, 2013, and a second season, titled Sword Art Online II, airing between July and December 2014. An animated film titled Sword Art Online the Movie: Ordinal Scale, featuring an original story by Kawahara, premiered in Japan and Southeast Asia on February 18, 2017, and was released in the United States on March 9, 2017. A spin-off anime series titled Sword Art Online Alternative: Gun Gale Online premiered in April 2018, while a third season titled Sword Art Online: Alicization aired from October 2018 to September 2020. An anime film adaptation of Sword Art Online: Progressive titled Sword Art Online Progressive: Aria of a Starless Night premiered on October 30, 2021. A second film titled Sword Art Online Progressive: Scherzo of Deep Night premiered on October 22, 2022. Many video games based on the series have been released for consoles, PC, and mobile devices. Sword Art Online has achieved widespread commercial success, with the light novels having over 30 million copies sold worldwide. The anime series has received mixed to positive reviews, with praise for its animation, musical score, and exploration of the psychological aspects of virtual reality, but it has also been met with criticisms for its pacing and writing. == Synopsis == === Setting === The light novel series spans several virtual reality worlds, beginning with the game, Sword Art Online (SAO), which is set in a world known as Aincrad. Each world is built on a game engine called Cardinal system, which was initially developed specifically for SAO by Akihiko Kayaba, but was later duplicated for Alfheim Online (ALO), and a consolidated package is later given to Kirito in the form of the World Seed, who had it leaked online with the successful intention of reviving the virtual reality industry. A third world known as Gun Gale Online (GGO) appears in the third arc and is stylized as a first-person shooter game instead of a role-playing game, and is the main setting of Alternative Gun Gale Online. It was created using the World Seed by an American company. A fourth world appears in the fourth arc known as the Underworld (UW). The world itself was created using the World Seed as a base, but it is as realistic as the real world due to using many powerful government resources to keep it running. === Plot === In 2022, a virtual reality massively multiplayer online role-playing game (VRMMORPG) called Sword Art Online (SAO) was released. With the NerveGear, a helmet that stimulates the user's five senses via their brain, players can experience and control their in-game characters with their minds. Both the game and the NerveGear were created by Akihiko Kayaba. On November 6, 10,000 players log into SAO's mainframe cyberspace for the first time, only to discover that they are unable to log out. Kayaba appears and tells the players that they must beat all 100 floors of Aincrad, a steel castle which is the setting of SAO, if they wish to be free. He also states that those who suffer in-game deaths or forcibly remove the NerveGear out-of-game will suffer real-life deaths. A player named Kazuto "Kirito" Kirigaya is one of 1,000 testers in the game's previous closed beta. With the advantage of previous VR gaming experience and a drive to protect other beta testers from discrimination, he isolates himself from the greater groups and plays the game alone, bearing the mantle of "beater", a portmanteau of "beta tester" and "cheater". As the players progress through the game Kirito eventually befriends a young woman named Asuna Yuuki, forming a relationship with and later marrying her in-game. After the duo discover the identity of Kayaba's secret ID, who was playing as "Heathcliff", the leader of the guild Asuna joined in, they confront and destroy him, freeing themselves and the other players from the game. In the real world, Kazuto discovers that 300 SAO players, including Asuna, remain trapped in their NerveGear. As he goes to the hospital to see Asuna, he meets Asuna's father Shouzou Yuuki who is asked by an associate of his, Nobuyuki Sugou, to make a decision, which Sugou later reveals to be his marriage with Asuna, angering Kazuto. Several months later, he is informed by Agil, another SAO survivor, that a figure similar to Asuna was spotted on "The World Tree" in another VRMMORPG cyberspace called Alfheim Online (ALO). Assisted in-game by his cousin and adoptive sister Suguha "Leafa" Kirigaya and Yui, a navigation pixie (originally an AI from SAO), he quickly learns that the trapped players in ALO are part of a plan conceived by Sugou to perform illegal experiments on their minds. The goal is to create the perfect mind-control for financial gain and to subjugate Asuna, whom he intends to marry in the real world, to assume control of her family's corporation. Kirito eventually stops the experiment and rescues the remaining 300 SAO players, foiling Sugou's plans. Before leaving ALO to see Asuna, Kayaba, who has uploaded his mind to the Internet using an experimental, destructively high-powered version of NerveGear at the cost of his life, entrusts Kirito with The Seed – a package program designed to create virtual worlds. Kazuto eventually reunites with Asuna in the real world after thwarting an attack from Sugou and The Seed is released onto the Internet, reviving Aincrad as other VRMMORPGs begin to thrive. One year after the events of SAO, at the prompting of a government official investigating strange occurrences in VR, Kazuto takes on a job to investigate a series of murders involving another VRMMORPG called Gun Gale Online (GGO), the AmuSphere (the successor of the NerveGear), and a player called Death Gun. Aided by a female player named Shino "Sinon" Asada, he participates in a gunfight tournament called the Bullet of Bullets (BoB) and discovers the truth behind the murders, which originated with a player who participated in a player-killing guild in SAO. Through his and Sinon's efforts, two suspects are captured, though the third suspect, Johnny Black, escapes. Kazuto is later recruited to test an experimental FullDive machine, Soul Translator (STL), which has an interface far more realistic and complex than the previous machine he had played, to help RATH, a research and development organization under the Ministry of Defense (MOD), develop an artificial intelligence named A.L.I.C.E. He tests the STL by entering the Underworld (UW), a virtual reality cyberspace created with The Seed package. In the UW, the flow of time proceeds a thousand times faster than in the real world, and Kirito's memories of what happens inside are restricted. However, when Johnny Black ambushes and mortally wounds Kazuto with suxamethonium chloride, RATH recovers Kazuto and places him back into the STL to preserve his mind while attempts are made to save him. During his time in Underworld, Kirito befriends Eugeo, a carver in a small village of Rulid, and helps him on a journey to save Alice Zuberg, his friend who was taken by a group of highly skilled warriors known as the Integrity Knights for accidentally breaking a rule of the Axiom Church, the leaders of the Human Empire. He and Eugeo soon find themselves uncovering the secrets of the Axiom Church, led by a woman only known as "The Administrator", and the true purpose of Underworld itself, while unbeknownst to them, a war against the opposing Dark Territory is brewing on the horizon. They meet Alice, now an Integrity Knight, and though she does not remember them, Kirito helps her remember her true identity: a form of true artificial intelligence known as A.L.I.C.E. In the battle against the Administrator, Kirito manages to slay her, though Eugeo dies in the process, to Kirito's dismay. Meanwhile, in the real world, conflict escalates as American forces raid RATH's facility in the Ocean Turtle in an effort to take A.L.I.C.E. for purposes unknown. Two of the attackers - Gabriel "Vecta" Miller and Vassago "Prince of Hell" Cassals - take contr

Void Trilogy

The Void Trilogy is a space opera series by British author Peter F. Hamilton. The series is set in the same universe as The Commonwealth Saga, 1,200 years after the end of Judas Unchained. Peter F. Hamilton sold the American rights to the series to Random House. The series includes the following books: The Dreaming Void (2007) The Temporal Void (2008) The Evolutionary Void (2010) == Synopsis == === The Dreaming Void === What was formerly believed to be a supermassive black hole at the centre of the Milky Way is revealed to be an artificial construct, known as the Void. Inside, there is a strange universe where the laws of physics are very different from standard physics. It is slowly consuming the other stars of the galactic core—one day it will have devoured the entire galaxy. In AD 3320, a human member of the Commonwealth, Inigo, begins to have dreams of the wonderful existence inside the Void. His dreams inspire the disaffected, who desire to travel into the Void, where their every wish will be fulfilled. By AD 3456, the pseudo-religious Living Dream movement exceeds 5 billion members, organizing the followers into a powerful political force. Other star-faring species fear their migration will cause the Void to expand again thus devouring the galaxy. They are prepared to stop the pilgrimage fleet no matter what the cost. The Dreaming Void is broken into two distinct sections. The first follows Edeard, a young boy who lives inside the Void on a planet called Querencia, the subject of Inigo's dreams. Edeard, an orphan and apprentice, lives in Ashwell, a town in Rulan province. A gifted psychic, he is trained by Master Akeem in crafting and modding. Initially a loner, he comes to prominence in his village after designing an alternative pump mechanism for the local well. Unfortunately his luck changes for the worse after Ashwell is raided by bandits. Forced to flee, he joins the local caravan and travels to Makkathran, the capital of Querencia. In Makkathran, Edeard joins the constables and after a brutal couple of months in training, he graduates and is promoted to the commander of his Squad. He makes little progress battling the rigid and backward judicial system of Makkathran; his first real break is when his squad overcomes a trap set by the local gang, and Edeard walks on water chasing the leader of the gang. A testament to his growing psychic abilities, Edeard's stunt earns him the title of Waterwalker, and he becomes an instant star in Makkathran. The second section of The Dreaming Void is set back in the Commonwealth. Inigo, the first dreamer, and founder of Living Dream, has disappeared, leaving the 5 billion strong Living Dream movement in a state of flux. When Ethan, succeeding Inigo as the head of the movement, proclaims that the Living Dream will embark on a pilgrimage into the Void, the Commonwealth is thrown into a state of political chaos. Fearing that the human migration might cause the Void to expand (and in the process destroy whole systems or even the whole Galaxy) other spacefaring races such as the Raiel and Ocisen Empire are deeply concerned, with the latter threatening military action. This has left the Commonwealth government deeply divided, with the two largest factions in disagreement, the Accelerators faction/party supporting the pilgrimage and the Conservative faction opposing. As both parties are unable to solve the situation politically they have resolved to take matters into their own hands, with each party sending agents to further its interests. Aaron, a sleeper cell agent, is tasked with finding Inigo. He kidnaps and manipulates Corrie-Lyn, a former lover of Inigo and interrogates her for information. He also travels to Kuhmo (Inigo's homeworld) to get further information and robs Inigo's secure storage (a bank for memory). He eventually tracks Inigo to Hanko, a desolate and barren world. However, before Aaron can extract Inigo, Accelerator agents destroy Aaron's starship leaving him marooned on Hanko. Meanwhile, Accelerator agents make a deal with Ethan, agreeing to give the Living Dream movement Ultra Drives to power their ships. Accelerator plans are halted when the Delivery Man, a Conservative party agent, destroys valuable FTL Drive tech. Troblum, an Accelerator physicist, also defects, further slowing the Accelerators plans. === The Temporal Void === The Temporal Void picks up after The Dreaming Void. The Intersolar Commonwealth faces mounting turmoil as the deadline for Living Dream's Pilgrimage into the Void approaches. An Ocisen Empire fleet advances on a mission of genocide, while an internecine war erupts among post-human factions over humanity's future. Amidst the chaos, investigator Paula Myo struggles to counter the increasingly desperate actions of various agents and factions. Relentless in her pursuit, she contends with adversaries from her distant past and colleagues of uncertain loyalty, all while racing against time. At the center of the unfolding crisis is Edeard the Waterwalker, a figure from the distant past who lived deep within the Void. As the messiah of Living Dream, his life—broadcast through visions—captivates and inspires billions. His story fuels the Pilgrimage's momentum, a force seemingly impossible to stop. As Edeard approaches his ultimate victory, the true nature of the Void is finally revealed. === The Evolutionary Void === The Evolutionary Void picks up after The Temporal Void. Exposed as the Second Dreamer, Araminta has become the target of a galaxy-wide search by government agent Paula Myo and the psychopath known as the Cat, along with others equally determined to prevent, or facilitate, the pilgrimage of the Living Dream cult into the heart of the Void. An indestructible microuniverse, the Void may contain paradise, as the cultists believe, but it is also a deadly threat. For the miraculous reality that exists inside its boundaries demands energy, energy drawn from everything outside those boundaries: from planets, stars, galaxies, and everything that lives, for the Pilgrimage will trigger a super-massive expansion of the Void. Meanwhile, the parallel story of Edeard, the Waterwalker, as told through a series of dreams communicated to the gaiafield via Inigo, the First Dreamer, continues to unfold. But the inspirational tale of this idealistic young man takes a darker and more troubling turn as he finds himself faced with powerful new enemies, and temptations more powerful still, to reach fulfilment in the end. Named a Silfen Friend like her ancestress Mellanie, Araminta chooses to face her unwanted responsibilities, with no guarantee of success or survival. She takes on the role of Second Dreamer to lead the first wave of Living Dream, 24 million people, into the Void, leaving everyone confused and lost by her actions. However, in actuality, she is playing a double game. Using her original body to lead the Living Dream as a diversion, she borrows one of her fiancé's (Mr. Bovey) bodies to set out to destroy the Void. She is able to connect with a Skylord and travel the Silfen Paths. With time running out, a repentant Inigo decides to release Edeard's final dream whose message is scarcely less dangerous than the pilgrimage promises to be, where perfection is achieved, so that nothing else is left to strive for and the human race in the Void has started to devolve. He goes to the Spike to meet Ozzie and stays there to meet with Araminta, who is using one of her fiancé's bodies, and Oscar. Third Dreamer Gore Burnelli has a plan to reason with the Heart, the core of the Void. He secures the help of the Delivery Man and travels to the Anomine homeworld to retrieve the mechanism that allowed them to go post-physical. He is able to connect with Justine, his daughter, who is currently in the Void, by way of Dreams. The monomaniacal Ilanthe, leader of the breakaway Accelerator Faction, seeks dominion in the Void. It is not Fusion with the Void to attain post-physical status that she wants, but to have control over everything. Using Dark Fortress technology, she sets up a barrier around the Sol system which leaves ANA and the deterrence fleet trapped inside. It is this technology which she has equipped the ships travelling to the Void with, the ability to create a forcefield which the Warrior Raiel cannot penetrate. == Technology == The Commonwealth uses a number of advanced technologies. In the early days of the Commonwealth, humans used static and permanently opened wormholes to travel from planet to planet. However, after the events of the Starflyer War (detailed in the Commonwealth Saga), the CST corporation's monopoly on space travel was ended. With the advent of wormholes that could wrap around ships, the Commonwealth saw a shift from wormholes to spaceships. Another development in the Commonwealth is the gaiafield. Developed by Ozzie Issac in AD 3000, the gaiafield is based on Silfen technology; when Ozzie was named a friend of the Silfen during the Starflye

GeoNetwork opensource

The GeoNetwork opensource (GNOS) project is a free and open source (FOSS) cataloging application for spatially referenced resources. It is a catalog of location-oriented information. == Outline == It is a standardized and decentralized spatial information management environment designed to enable access to geo-referenced databases, cartographic products and related metadata from a variety of sources, enhancing the spatial information exchange and sharing between organizations and their audience, using the capacities of the internet. Using the Z39.50 protocol it both accesses remote catalogs and makes its data available to other catalog services. As of 2007, OGC Web Catalog Service are being implemented. Maps, including those derived from satellite imagery, are effective communicational tools and play an important role in the work of decision makers (e.g., sustainable development planners and humanitarian and emergency managers) in need of quick, reliable and up-to-date user-friendly cartographic products as a basis for action and to better plan and monitor their activities; GIS experts in need of exchanging consistent and updated geographical data; and spatial analysts in need of multidisciplinary data to perform preliminary geographical analysis and make reliable forecasts. == Deployment == The software has been deployed to various organizations, the first being FAO GeoNetwork and WFP VAM-SIE-GeoNetwork, both at their headquarters in Rome, Italy. Furthermore, the WHO, CGIAR, BRGM, ESA, FGDC and the Global Change Information and Research Centre (GCIRC) of China are working on GeoNetwork opensource implementations as their spatial information management capacity. It is used for several risk information systems, in particular in the Gambia. Several related tools are packaged with GeoNetwork, including GeoServer. GeoServer stores geographical data, while GeoNetwork catalogs collections of such data.

Recraft

Recraft is a generative artificial intelligence program and service developed by the London-based startup Recraft, Inc. The company also offers Recraft Studio, a web-based workspace that lets users create and edit images, vectors, and mockups using various text-to-image models. Like models such as Midjourney and DALL-E, the Recraft model generates digital images from natural language prompts, and is specifically tailored for creative workflows, with features that emphasize brand consistency, text fidelity, and layout control. == History and background == Recraft, Inc. was founded in 2022 by machine learning scientist Anna Veronika Dorogush, best known for co-creating the CatBoost machine learning library at Yandex. The company emerged from stealth on May 31, 2023, with a public release of its vector graphics generation capability on Product Hunt. On January 17, 2024, TechCrunch profiled Recraft’s foundational model for graphic design, noting its emphasis on addressing copyright and ethical concerns associated with AI-generated imagery. On October 28, 2024, TechCrunch reported that Recraft's third major model, V3, had topped a crowdsourced benchmark, surpassing Midjourney and OpenAI's DALL-E in overall image quality. On May 5, 2025, Recraft announced a $30 million Series B funding round led by Accel, reporting more than four million registered users at the time of the announcement. == Models == Recraft has developed multiple generations of its text-to-image models since 2022. Each generation reflects improvements in fidelity, controllability, and support for both raster and vector outputs. The models are proprietary and accessible through the Recraft API, Recraft Studio. Recraft models are also hosted as an image generation API on fal, Replicate, Prodia, and others. === Recraft V2 === Recraft V2 was released in March 2024 and was the company’s first model trained from scratch. It contained roughly 20 billion parameters and introduced native vector image generation, brand-color conditioning, and improved stylistic consistency for icons and illustrations. === Recraft V3 === Recraft V3 was released in October 2024 and achieved first place on the Artificial Analysis benchmark hosted on Hugging Face. The model introduced advances in photorealism, improved rendering of multi-word text, and increased responsiveness to detailed descriptive prompts. It also added the “Artistic” parameter, which allowed users to adjust stylistic intensity within generated images. === Recraft V4 === Recraft V4 was released in February 2026. According to Recraft, V4 is a “ground-up rebuild” aimed at improving prompt accuracy and output quality for design workflows, with the company emphasizing “design taste” and art-directed results. Recraft states that V4 is available in two versions: V4 for faster iteration and V4 Pro for higher-resolution, print-ready assets; the API documentation describes V4 as 1-megapixel output and V4 Pro as 4-megapixel output, with vector variants available for each. === Features === Vectorization: Recraft’s models can generate and convert images into native vector formats, producing scalable graphics composed of editable paths rather than fixed pixels. Style reference: The models support the use of reference images to guide stylistic characteristics such as color palette, line quality, composition, or visual tone. Style mixing: Recraft models can combine multiple stylistic inputs within a single generation. By blending attributes from different references or stylistic instructions, the system produces images that reflect hybrid visual characteristics while maintaining internal consistency. Inpainting editing: The models support localized image modification through inpainting, enabling users to regenerate selected regions of an image while preserving surrounding content. === Model capabilities === Recraft’s models generate raster and vector images from natural-language prompts and are designed to interpret detailed descriptions with attention to composition, style, and text placement. The models support controlled stylistic variation through preset or reference-based guidance and can maintain coherent line, color, or layout structure across multiple outputs. They produce scalable vector graphics alongside high-resolution raster images, and include features for localized image modification through inpainting or outpainting operations. === Technology === Recraft has not publicly disclosed the detailed technical architecture of its models. However, third-party reviews and benchmarks have noted that its performance resembles diffusion models such as Midjourney and Stable Diffusion. The model is designed for creative workflows requiring visual consistency and flexible output formats. Reviewers have noted its ability to generate legible multi-line text, produce high-resolution imagery at various canvas sizes, and to maintain alignment with user-defined brand palettes and design themes. Though not open-source, Recraft's models are accessible through a web interface and commercial API. Advanced features such as style settings and positioning control differentiate it from general-purpose text-to-image models. == Recraft Studio == Recraft Studio is a web-based workspace for generating and editing images using Recraft’s image models and selected external models. The infinite canvas interface provides access to a range of creation and refinement tools within a single environment. Raster and vector generation with styles: Recraft Studio supports the generation of both raster and vector images. Users can apply predefined or reference-based styles during generation, allowing for visual consistency across multiple outputs. Mockups: The studio includes mockup tools that allow generated designs to be placed onto predefined surfaces or templates for visualization and presentation purposes. Vectorization: Recraft Studio provides vectorization tools that convert raster images into editable vector graphics, enabling further modification of shapes, colors, and layout. Image upscaling: The workspace includes image upscaling functionality for increasing resolution while preserving visual detail. Editing tools and natural-language editing: Recraft Studio offers a set of editing tools for modifying images within the canvas, including localized adjustments and natural-language–based editing commands that allow users to describe changes using text. === Supported models === Recraft Studio provides access to Recraft’s proprietary image models as well as other external frontier image models such as Nano Banana, GPT 4-o, Imagen, Flux, and others. == Business model == Recraft develops proprietary image models that are accessible through Recraft Studio and the Recraft API. Recraft Studio operates on a freemium model, offering a free tier with limited daily credits and paid subscriptions for access to additional features. The API follows a credit-based system in which units are purchased separately for programmatic image generation. A team plan supports collaborative use, and the API enables organizations and developers to integrate Recraft’s image generation and editing capabilities into their own systems and workflows.