Glossary of computer graphics

Glossary of computer graphics

This is a glossary of terms relating to computer graphics. For more general computer hardware terms, see glossary of computer hardware terms. == 0–9 == 2D convolution Operation that applies linear filtering to image with a given two-dimensional kernel, able to achieve e.g. edge detection, blurring, etc. 2D image 2D texture map A texture map with two dimensions, typically indexed by UV coordinates. 2D vector A two-dimensional vector, a common data type in rasterization algorithms, 2D computer graphics, graphical user interface libraries. 2.5D Also pseudo 3D. Rendering whose result looks 3D while actually not being 3D or having great limitations, e.g. in camera degrees of freedom. 3D graphics pipeline A graphics pipeline taking 3D models and producing a 2D bitmap image result. 3D paint tool A 3D graphics application for digital painting of multiple texture map image channels directly onto a rotated 3D model, such as zbrush or mudbox, also sometimes able to modify vertex attributes. 3D scene A collection of 3D models and lightsources in world space, into which a camera may be placed, describing a scene for 3D rendering. 3D unit vector A unit vector in 3D space. 4D vector A common datatype in graphics code, holding homogeneous coordinates or RGBA data, or simply a 3D vector with unused W to benefit from alignment, naturally handled by machines with 4-element SIMD registers. 4×4 matrix A matrix commonly used as a transformation of homogeneous coordinates in 3D graphics pipelines. 7e3 format A packed pixel format supported by some graphics processing units (GPUs) where a single 32-bit word encodes three 10-bit floating-point color channels, each with seven bits of mantissa and three bits of exponent. == A == AABB Axis-aligned bounding box (sometimes called "axis oriented"), a bounding box stored in world coordinates; one of the simplest bounding volumes. Additive blending A compositing operation where d s t = d s t + s r c , {\displaystyle dst=dst+src,} without the use of an alpha channel, used for various effects. Also known as linear dodge in some applications. Affine texture mapping Linear interpolation of texture coordinates in screen space without taking perspective into account, causing texture distortion. Aliasing Unwanted effect arising when sampling high-frequency signals, in computer graphics appearing e.g. when downscaling images. Antialiasing methods can prevent it. Alpha channel An additional image channel (e.g. extending an RGB image) or standalone channel controlling alpha blending. Ambient lighting An approximation to the light entering a region from a wide range of directions, used to avoid needing an exact solution to the rendering equation. Ambient occlusion (AO) Effect approximating, in an inexpensive way, one aspect of global illumination by taking into account how much ambient light is blocked by nearby geometry, adding visual clues about the shape. Analytic model A mathematical model for a phenomenon to be simulated, e.g. some approximation to surface shading. Contrasts with Empirical models based purely on recorded data. Anisotropic filtering Advanced texture filtering improving on mipmapping, preventing aliasing while reducing blur in textured polygons at oblique angles to the camera. Anti-aliasing Methods for filtering and sampling to avoid visual artifacts associated with the uniform pixel grid in 3D rendering. Array texture A form of texture map containing an array of 2D texture slices selectable by a 3rd 'W' texture coordinate; used to reduce state changes in 3D rendering. Augmented reality Computer-rendered content inserted into the user's view of the real world. AZDO Approaching zero driver overhead, a set of techniques aimed at reducing the CPU overhead in preparing and submitting rendering commands in the OpenGL pipeline. A compromise between the traditional GL API and other high-performance low-level rendering APIs. == B == Back-face culling Culling (discarding) of polygons that are facing backwards from the camera. Baking Performing an expensive calculation offline, and caching the results in a texture map or vertex attributes. Typically used for generating lightmaps, normal maps, or low level of detail models. Barycentric coordinates Three-element coordinates of a point inside a triangle. Beam tracing Modification of ray tracing which instead of lines uses pyramid-shaped beams to address some of the shortcomings of traditional ray tracing, such as aliasing. Bicubic interpolation Extension of cubic interpolation to 2D, commonly used when scaling textures. Bilinear interpolation Linear interpolation extended to 2D, commonly used when scaling textures. Binding Selecting a resource (texture, buffer, etc.) to be referenced by future commands. Billboard A textured rectangle that keeps itself oriented towards the camera, typically used e.g. for vegetation or particle effects. Binary space partitioning (BSP) A data structure that can be used to accelerate visibility determination, used e.g. in Doom engine. Bit depth The number of bits per pixel, sample, or texel in a bitmap image (holding one or more image channels, typical values being 4, 8, 16, 24, 32) Bitmap Image stored by pixels. Bit plane A format for bitmap images storing 1 bit per pixel in a contiguous 2D array; Several such parallel arrays combine to produce the a higher-bit-depth image. Opposite of packed-pixel format. Blend operation A render state controlling alpha blending, describing a formula for combining source and destination pixels. Bone Coordinate systems used to control surface deformation (via Weight maps) during skeletal animation. Typically stored in a hierarchy, controlled by key frames, and other procedural constraints. Bounding box One of the simplest type of bounding volume, consisting of axis-aligned or object-aligned extents. Bounding volume A mathematically simple volume, such as a sphere or a box, containing 3D objects, used to simplify and accelerate spatial tests (e.g. for visibility or collisions). BRDF Bidirectional reflectance distribution functions (BRDFs), empirical models defining 4D functions for surface shading indexed by a view vector and light vector relative to a surface. Bump mapping Technique similar to normal mapping that instead of normal maps uses so called bump maps (height maps). BVH Bounding volume hierarchy is a tree structure on a set of geometric objects. == C == Camera A virtual camera from which rendering is performed, also sometimes referred to as 'eye'. Camera space A space with the camera at the origin, aligned with the viewer's direction, after the application of the world transformation and view transformation. Cel shading Cartoon-like shading effect. Clipping Limiting specific operations to a specific region, usually the view frustum. Clipping plane A plane used to clip rendering primitives in a graphics pipeline. These may define the view frustum or be used for other effects. Clip space Coordinate space in which clipping is performed. Clip window A rectangular region in screen space, used during clipping. A clip window may be used to enclose a region around a portal in portal rendering. CLUT A table of RGB color values to be indexed by a lower-bit-depth image (typically 4–8 bits), a form of vector quantization. Color bleeding Unwanted effect in texture mapping. A color from a border of unmapped region of the texture may appear (bleed) in the mapped result due to interpolation. Color channels The set of channels in a bitmap image representing the visible color components, i.e. distinct from the alpha channel or other information. Color resolution Command buffer A region of memory holding a set of instructions for a graphics processing unit for rendering a scene or portion of a scene. These may be generated manually in bare metal programming, or managed by low level rendering APIs, or handled internally by high level rendering APIs. Command list A group of rendering commands ready for submission to a graphics processing unit, see also Command buffer. Compute API An API for efficiently processing large amounts of data. Compute shader A compute kernel managed by a rendering API, with easy access to rendering resources. Cone tracing Modification of ray tracing which instead of lines uses cones as rays in order to achieve e.g. antialiasing or soft shadows. Connectivity information Indices defining [rendering primitive]s between vertices, possibly held in index buffers. describes geometry as a graph or hypergraph. CSG Constructive solid geometry, a method for generating complex solid models from boolean operations combining simpler modelling primitives. Cube mapping A form of environment reflection mapping in which the environment is captured on a surface of a cube (cube map). Culling Before rendering begins, culling removes objects that don't significantly contribute to the rendered result (e.g. being obscured or outside camera view). == D == Decal A "sticker" picture applied onto a surface (e.g. a

Socially assistive robot

A socially assistive robot (SAR) aids users through social engagement and support rather than through physical tasks and interactions. == Background == The field of socially assistive robotics emerged in the early 2000s, following the emergence of the field of social robots. In contrast to social robots, SARs aid users with specific goals related to behavior change rather than serving as purely social entities. The term "Socially assistive robot" was initially defined by Maja Matarić and David Feil-Seifer in 2005. Since its inception, the field has gained substantial recognition, featuring numerous research projects, a wealth of global research publications, startup companies, and a growing array of products on the consumer market. The COVID-19 pandemic has underscored the immense potential of socially assistive robots, particularly in addressing the needs of large user populations, including children engaged in remote learning, elderly individuals grappling with loneliness, and those affected by social isolation and its associated negative consequences. == Characteristics of interaction == SARs rely on artificial intelligence (AI) to generate real-time, responsive, natural, and meaningful robot behaviors during interactions with humans. The robots employ various forms of communication, such as facial expressions, gestures, body movements, and speech. In contrast to robots intended for physical tasks, SARs are designed to support and motivate users to perform their own tasks. The tasks a user engages in can be physical (e.g., rehabilitation exercises for post-stroke users), cognitive (e.g., dementia screening for elderly users), or social (e.g., turn-taking for users with autism spectrum disorders). This complex interaction involves detecting and interpreting the user's movement, behavior, intent, goals, speech, and preferences. Machine learning and robot learning techniques are frequently employed to enhance the robot's understanding of the user, predict user preferences, and provide effective assistance. The effectiveness of socially assistive robots is assessed based on objective measurements of user performance and improvement resulting from the robot’s assistance and support. Unlike other branches of robotics, where effectiveness depends on the robot's physical task completion, SAR measures the success of the robot based on the user's progress and achievements. This evaluation is carried out using quantitative objective metrics, such as time spent on tasks, accuracy, retention, and verbalization, as well as quantitative subjective metrics, such as user survey tools. SAR is based on the large body of evidence showing that users tend to respond more positively to interactions with physical robots compared to interactions with screens. Interaction with physical robots also encourages users to learn and retain more information than screen-based interactions. This fundamental insight underlines why physical robots in SAR applications are more effective, as opposed to interactions solely involving screens, tablets, or computers. == Uses and applications == SARs have been developed and validated in a wide array of applications, including healthcare, elder care, education, and training. For example, SARs have been developed to support children on the autism spectrum in acquiring and practicing social and cognitive skills, to motivate and coach stroke patients throughout their rehabilitation exercises, monitoring individuals health (ex. fall detection), and to encourage elderly users to be more physically and socially active. There is a concern that technophobia and lack of trust in robots will pose a barrier to the effectiveness of SARs in older adults.

Storage area network

A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from servers so that the devices appear to the operating system as direct-attached storage. A SAN typically is a dedicated network of storage devices not accessible through the local area network (LAN). Although a SAN provides only block-level access, file systems built on top of SANs do provide file-level access and are known as shared-disk file systems. Newer SAN configurations enable hybrid SAN and allow traditional block storage that appears as local storage but also object storage for web services through APIs. == Storage architectures == Storage area networks (SANs) are sometimes referred to as network behind the servers and historically developed out of a centralized data storage model, but with its own data network. A SAN is, at its simplest, a dedicated network for data storage. In addition to storing data, SANs allow for the automatic backup of data, and the monitoring of the storage as well as the backup process. A SAN is a combination of hardware and software. It grew out of data-centric mainframe architectures, where clients in a network can connect to several servers that store different types of data. To scale storage capacities as the volumes of data grew, direct-attached storage (DAS) was developed, where disk arrays or just a bunch of disks (JBODs) were attached to servers. In this architecture, storage devices can be added to increase storage capacity. However, the server through which the storage devices are accessed is a single point of failure, and a large part of the LAN network bandwidth is used for accessing, storing and backing up data. To solve the single point of failure issue, a direct-attached shared storage architecture was implemented, where several servers could access the same storage device. DAS was the first network storage system and is still widely used where data storage requirements are not very high. Out of it developed the network-attached storage (NAS) architecture, where one or more dedicated file server or storage devices are made available in a LAN. Therefore, the transfer of data, particularly for backup, still takes place over the existing LAN. If more than a terabyte of data was stored at any one time, LAN bandwidth became a bottleneck. Therefore, SANs were developed, where a dedicated storage network was attached to the LAN, and terabytes of data are transferred over a dedicated high speed and bandwidth network. Within the SAN, storage devices are interconnected. Transfer of data between storage devices, such as for backup, happens behind the servers and is meant to be transparent. In a NAS architecture data is transferred using the TCP and IP protocols over Ethernet. Distinct protocols were developed for SANs, such as Fibre Channel, iSCSI, Infiniband. Therefore, SANs often have their own network and storage devices, which have to be bought, installed, and configured. This makes SANs inherently more expensive than NAS architectures. == Components == SANs have their own networking devices, such as SAN switches. To access the SAN, so-called SAN servers are used, which in turn connect to SAN host adapters. Within the SAN, a range of data storage devices may be interconnected, such as SAN-capable disk arrays, JBODs and tape libraries. === Host layer === Servers that allow access to the SAN and its storage devices are said to form the host layer of the SAN. Such servers have host adapters, which are cards that attach to slots on the server motherboard (usually PCI slots) and run with a corresponding firmware and device driver. Through the host adapters the operating system of the server can communicate with the storage devices in the SAN. In Fibre channel deployments, a cable connects to the host adapter through the gigabit interface converter (GBIC). GBICs are also used on switches and storage devices within the SAN, and they convert digital bits into light impulses that can then be transmitted over the Fibre Channel cables. Conversely, the GBIC converts incoming light impulses back into digital bits. The predecessor of the GBIC was called gigabit link module (GLM). === Fabric layer === The fabric layer consists of SAN networking devices that include SAN switches, routers, protocol bridges, gateway devices, and cables. SAN network devices move data within the SAN, or between an initiator, such as an HBA port of a server, and a target, such as the port of a storage device. When SANs were first built, hubs were the only devices that were Fibre Channel capable, but Fibre Channel switches were developed and hubs are now rarely found in SANs. Switches have the advantage over hubs that they allow all attached devices to communicate simultaneously, as a switch provides a dedicated link to connect all its ports with one another. When SANs were first built, Fibre Channel had to be implemented over copper cables, these days multimode optical fibre cables are used in SANs. SANs are usually built with redundancy, so SAN switches are connected with redundant links. SAN switches connect the servers with the storage devices and are typically non-blocking allowing transmission of data across all attached wires at the same time. SAN switches are for redundancy purposes set up in a meshed topology. A single SAN switch can have as few as 8 ports and up to 32 ports with modular extensions. So-called director-class switches can have as many as 128 ports. In switched SANs, the Fibre Channel switched fabric protocol FC-SW-6 is used under which every device in the SAN has a hardcoded World Wide Name (WWN) address in the host bus adapter (HBA). If a device is connected to the SAN its WWN is registered in the SAN switch name server. In place of a WWN, or worldwide port name (WWPN), SAN Fibre Channel storage device vendors may also hardcode a worldwide node name (WWNN). The ports of storage devices often have a WWN starting with 5, while the bus adapters of servers start with 10 or 21. === Storage layer === The serialized Small Computer Systems Interface (SCSI) protocol is often used on top of the Fibre Channel switched fabric protocol in servers and SAN storage devices. The Internet Small Computer Systems Interface (iSCSI) over Ethernet and the Infiniband protocols may also be found implemented in SANs, but are often bridged into the Fibre Channel SAN. However, Infiniband and iSCSI storage devices, in particular, disk arrays, are available. The various storage devices in a SAN are said to form the storage layer. It can include a variety of hard disk and magnetic tape devices that store data. In SANs, disk arrays are joined through a RAID which makes a lot of hard disks look and perform like one big storage device. Every storage device, or even partition on that storage device, has a logical unit number (LUN) assigned to it. This is a unique number within the SAN. Every node in the SAN, be it a server or another storage device, can access the storage by referencing the LUN. The LUNs allow for the storage capacity of a SAN to be segmented and for the implementation of access controls. A particular server, or a group of servers, may, for example, be only given access to a particular part of the SAN storage layer, in the form of LUNs. When a storage device receives a request to read or write data, it will check its access list to establish whether the node, identified by its LUN, is allowed to access the storage area, also identified by a LUN. LUN masking is a technique whereby the host bus adapter and the SAN software of a server restrict the LUNs for which commands are accepted. In doing so LUNs that should never be accessed by the server are masked. Another method to restrict server access to particular SAN storage devices is fabric-based access control, or zoning, which is enforced by the SAN networking devices and servers. Under zoning, server access is restricted to storage devices that are in a particular SAN zone. == Network protocols == A mapping layer to other protocols is used to form a network: ATA over Ethernet (AoE), mapping of AT Attachment (ATA) over Ethernet Fibre Channel Protocol (FCP), a mapping of SCSI over Fibre Channel Fibre Channel over Ethernet (FCoE) ESCON over Fibre Channel (FICON), used by mainframe computers HyperSCSI, mapping of SCSI over Ethernet iFCP or SANoIP mapping of FCP over IP iSCSI, mapping of SCSI over TCP/IP iSCSI Extensions for RDMA (iSER), mapping of iSCSI over InfiniBand Network block device, mapping device node requests on UNIX-like systems over stream sockets like TCP/IP SCSI RDMA Protocol (SRP), another SCSI implementation for remote direct memory access (RDMA) transports Storage networks may also be built using Serial Attached SCSI (SAS) and Serial ATA (SATA) technologies. SAS evolved from SCSI direct-attached storage. SATA evolved from Para

Tagsistant

Tagsistant is a semantic file system for the Linux kernel, written in C and based on FUSE. Unlike traditional file systems that use hierarchies of directories to locate objects, Tagsistant introduces the concept of tags. == Design and differences with hierarchical file systems == In computing, a file system is a type of data store which could be used to store, retrieve and update files. Each file can be uniquely located by its path. The user must know the path in advance to access a file and the path does not necessarily include any information about the content of the file. Tagsistant uses a complementary approach based on tags. The user can create a set of tags and apply those tags to files, directories and other objects (devices, pipes, ...). The user can then search all the objects that match a subset of tags, called a query. This kind of approach is well suited for managing user contents like pictures, audio recordings, movies and text documents but is incompatible with system files (like libraries, commands and configurations) where the univocity of the path is a security requirement to prevent the access to a wrong content. == The tags/ directory == A Tagsistant file system features four main directories: archive/ relations/ stats/ tags/ Tags are created as sub directories of the tags/ directory and can be used in queries complying to this syntax: tags/subquery/[+/subquery/[+/subquery/]]/@/ where a subquery is an unlimited list of tags, concatenated as directories: tag1/tag2/tag3/.../tagN/ The portion of a path delimited by tags/ and @/ is the actual query. The +/ operator joins the results of different sub-queries in one single list. The @/ operator ends the query. To be returned as a result of the following query: tags/t1/t2/+/t1/t4/@/ an object must be tagged as both t1/ and t2/ or as both t1/ and t4/. Any object tagged as t2/ or t4/, but not as t1/ will not be retrieved. The query syntax deliberately violates the POSIX file system semantics by allowing a path token to be a descendant of itself, like in tags/t1/t2/+/t1/t4/@ where t1/ appears twice. As a consequence a recursive scan of a Tagsistant file system will exit with an error or endlessly loop, as done by Unix find: This drawback is balanced by the possibility to list the tags inside a query in any order. The query tags/t1/t2/@/ is completely equivalent to tags/t2/t1/@/ and tags/t1/+/t2/t3/@/ is equivalent to tags/t2/t3/+/t1/@/. The @/ element has the precise purpose of restoring the POSIX semantics: the path tags/t1/@/directory/ refers to a traditional directory and a recursive scan of this path will properly perform. == The reasoner and the relations/ directory == Tagsistant features a simple reasoner which expands the results of a query by including objects tagged with related tags. A relation between two tags can be established inside the relations/ directory following a three level pattern: relations/tag1/rel/tag2/ The rel element can be includes or is_equivalent. To include the rock tag in the music tag, the Unix command mkdir can be used: mkdir -p relations/music/includes/rock The reasoner can recursively resolve relations, allowing the creation of complex structures: mkdir -p relations/music/includes/rock mkdir -p relations/rock/includes/hard_rock mkdir -p relations/rock/includes/grunge mkdir -p relations/rock/includes/heavy_metal mkdir -p relations/heavy_metal/includes/speed_metal The web of relations created inside the relations/ directory constitutes a basic form of ontology. == Autotagging plugins == Tagsistant features an autotagging plugin stack which gets called when a file or a symlink is written. Each plugin is called if its declared MIME type matches The list of working plugins released with Tagsistant 0.6 is limited to: text/html: tags the file with each word in and <keywords> elements and with document, webpage and html too image/jpeg: tags the file with each Exif tag == The repository == Each Tagsistant file system has a corresponding repository containing an archive/ directory where the objects are actually saved and a tags.sql file holding tagging information as an SQLite database. If the MySQL database engine was specified with the --db argument, the tags.sql file will be empty. Another file named repository.ini is a GLib ini store with the repository configuration. Tagsistant 0.6 is compatible with the MySQL and Sqlite dialects of SQL for tag reasoning and tagging resolution. While porting its logic to other SQL dialects is possible, differences in basic constructs (especially the INTERSECT SQL keyword) must be considered. == The archive/ and stats/ directories == The archive/ directory has been introduced to provide a quick way to access objects without using tags. Objects are listed with their inode number prefixed. The stats/ directory features some read-only files containing usage statistics. A file configuration holds both compile time information and current repository configuration. == Main criticisms == It has been highlighted that relying on an external database to store tags and tagging information could cause the complete loss of metadata if the database gets corrupted. It has been highlighted that using a flat namespace tends to overcrowd the tags/ directory. This could be mitigated introducing triple tags.</p> <h2><a href="https://aizhi.co/news/323c799669.html" title="Adaptive algorithm">Adaptive algorithm</a></h2> <p>An adaptive algorithm is an algorithm that changes its behavior at the time it is run, based on information available and on a priori defined reward mechanism (or criterion). Such information could be the story of recently received data, information on the available computational resources, or other run-time acquired (or a priori known) information related to the environment in which it operates. Among the most used adaptive algorithms is the Widrow-Hoff’s least mean squares (LMS), which represents a class of stochastic gradient-descent algorithms used in adaptive filtering and machine learning. In adaptive filtering the LMS is used to mimic a desired filter by finding the filter coefficients that relate to producing the least mean square of the error signal (difference between the desired and the actual signal). For example, stable partition, using no additional memory is O(n lg n) but given O(n) memory, it can be O(n) in time. As implemented by the C++ Standard Library, stable_partition is adaptive and so it acquires as much memory as it can get (up to what it would need at most) and applies the algorithm using that available memory. Another example is adaptive sort, whose behavior changes upon the presortedness of its input. An example of an adaptive algorithm in radar systems is the constant false alarm rate (CFAR) detector. In machine learning and optimization, many algorithms are adaptive or have adaptive variants, which usually means that the algorithm parameters such as learning rate are automatically adjusted according to statistics about the optimisation thus far (e.g. the rate of convergence). Examples include adaptive simulated annealing, adaptive coordinate descent, adaptive quadrature, AdaBoost, Adagrad, Adadelta, RMSprop, and Adam. In data compression, adaptive coding algorithms such as Adaptive Huffman coding or Prediction by partial matching can take a stream of data as input, and adapt their compression technique based on the symbols that they have already encountered. In signal processing, the Adaptive Transform Acoustic Coding (ATRAC) codec used in MiniDisc recorders is called "adaptive" because the window length (the size of an audio "chunk") can change according to the nature of the sound being compressed, to try to achieve the best-sounding compression strategy.</p> <h2><a href="https://aizhi.co/html/7b499988.html" title="Actor-critic algorithm">Actor-critic algorithm</a></h2> <p>The actor-critic algorithm (AC) is a family of reinforcement learning (RL) algorithms that combine policy-based RL algorithms such as policy gradient methods, and value-based RL algorithms such as value iteration, Q-learning, SARSA, and TD learning. An AC algorithm consists of two main components: an "actor" that determines which actions to take according to a policy function, and a "critic" that evaluates those actions according to a value function. Some AC algorithms are on-policy, some are off-policy. Some apply to either continuous or discrete action spaces. Some work in both cases. == Overview == The actor-critic methods can be understood as an improvement over pure policy gradient methods like REINFORCE via introducing a baseline. === Actor === The actor uses a policy function π ( a | s ) {\displaystyle \pi (a|s)} , while the critic estimates either the value function V ( s ) {\displaystyle V(s)} , the action-value Q-function Q ( s , a ) , {\displaystyle Q(s,a),} the advantage function A ( s , a ) {\displaystyle A(s,a)} , or any combination thereof. The actor is a parameterized function π θ {\displaystyle \pi _{\theta }} , where θ {\displaystyle \theta } are the parameters of the actor. The actor takes as argument the state of the environment s {\displaystyle s} and produces a probability distribution π θ ( ⋅ | s ) {\displaystyle \pi _{\theta }(\cdot |s)} . If the action space is discrete, then ∑ a π θ ( a | s ) = 1 {\displaystyle \sum _{a}\pi _{\theta }(a|s)=1} . If the action space is continuous, then ∫ a π θ ( a | s ) d a = 1 {\displaystyle \int _{a}\pi _{\theta }(a|s)da=1} . The goal of policy optimization is to improve the actor. That is, to find some θ {\displaystyle \theta } that maximizes the expected episodic reward J ( θ ) {\displaystyle J(\theta )} : J ( θ ) = E π θ [ ∑ t = 0 T γ t r t ] {\displaystyle J(\theta )=\mathbb {E} _{\pi _{\theta }}\left[\sum _{t=0}^{T}\gamma ^{t}r_{t}\right]} where γ {\displaystyle \gamma } is the discount factor, r t {\displaystyle r_{t}} is the reward at step t {\displaystyle t} , and T {\displaystyle T} is the time-horizon (which can be infinite). The goal of policy gradient method is to optimize J ( θ ) {\displaystyle J(\theta )} by gradient ascent on the policy gradient ∇ J ( θ ) {\displaystyle \nabla J(\theta )} . As detailed on the policy gradient method page, there are many unbiased estimators of the policy gradient: ∇ θ J ( θ ) = E π θ [ ∑ 0 ≤ j ≤ T ∇ θ ln ⁡ π θ ( A j | S j ) ⋅ Ψ j | S 0 = s 0 ] {\displaystyle \nabla _{\theta }J(\theta )=\mathbb {E} _{\pi _{\theta }}\left[\sum _{0\leq j\leq T}\nabla _{\theta }\ln \pi _{\theta }(A_{j}|S_{j})\cdot \Psi _{j}{\Big |}S_{0}=s_{0}\right]} where Ψ j {\textstyle \Psi _{j}} is a linear sum of the following: ∑ 0 ≤ i ≤ T ( γ i R i ) {\textstyle \sum _{0\leq i\leq T}(\gamma ^{i}R_{i})} . γ j ∑ j ≤ i ≤ T ( γ i − j R i ) {\textstyle \gamma ^{j}\sum _{j\leq i\leq T}(\gamma ^{i-j}R_{i})} : the REINFORCE algorithm. γ j ∑ j ≤ i ≤ T ( γ i − j R i ) − b ( S j ) {\textstyle \gamma ^{j}\sum _{j\leq i\leq T}(\gamma ^{i-j}R_{i})-b(S_{j})} : the REINFORCE with baseline algorithm. Here b {\displaystyle b} is an arbitrary function. γ j ( R j + γ V π θ ( S j + 1 ) − V π θ ( S j ) ) {\textstyle \gamma ^{j}\left(R_{j}+\gamma V^{\pi _{\theta }}(S_{j+1})-V^{\pi _{\theta }}(S_{j})\right)} : TD(1) learning. γ j Q π θ ( S j , A j ) {\textstyle \gamma ^{j}Q^{\pi _{\theta }}(S_{j},A_{j})} . γ j A π θ ( S j , A j ) {\textstyle \gamma ^{j}A^{\pi _{\theta }}(S_{j},A_{j})} : Advantage Actor-Critic (A2C). γ j ( R j + γ R j + 1 + γ 2 V π θ ( S j + 2 ) − V π θ ( S j ) ) {\textstyle \gamma ^{j}\left(R_{j}+\gamma R_{j+1}+\gamma ^{2}V^{\pi _{\theta }}(S_{j+2})-V^{\pi _{\theta }}(S_{j})\right)} : TD(2) learning. γ j ( ∑ k = 0 n − 1 γ k R j + k + γ n V π θ ( S j + n ) − V π θ ( S j ) ) {\textstyle \gamma ^{j}\left(\sum _{k=0}^{n-1}\gamma ^{k}R_{j+k}+\gamma ^{n}V^{\pi _{\theta }}(S_{j+n})-V^{\pi _{\theta }}(S_{j})\right)} : TD(n) learning. γ j ∑ n = 1 ∞ λ n − 1 1 − λ ⋅ ( ∑ k = 0 n − 1 γ k R j + k + γ n V π θ ( S j + n ) − V π θ ( S j ) ) {\textstyle \gamma ^{j}\sum _{n=1}^{\infty }{\frac {\lambda ^{n-1}}{1-\lambda }}\cdot \left(\sum _{k=0}^{n-1}\gamma ^{k}R_{j+k}+\gamma ^{n}V^{\pi _{\theta }}(S_{j+n})-V^{\pi _{\theta }}(S_{j})\right)} : TD(λ) learning, also known as GAE (generalized advantage estimate). This is obtained by an exponentially decaying sum of the TD(n) learning terms. === Critic === In the unbiased estimators given above, certain functions such as V π θ , Q π θ , A π θ {\displaystyle V^{\pi _{\theta }},Q^{\pi _{\theta }},A^{\pi _{\theta }}} appear. These are approximated by the critic. Since these functions all depend on the actor, the critic must learn alongside the actor. The critic is learned by value-based RL algorithms. For example, if the critic is estimating the state-value function V π θ ( s ) {\displaystyle V^{\pi _{\theta }}(s)} , then it can be learned by any value function approximation method. Let the critic be a function approximator V ϕ ( s ) {\displaystyle V_{\phi }(s)} with parameters ϕ {\displaystyle \phi } . The simplest example is TD(1) learning, which trains the critic to minimize the TD(1) error: δ i = R i + γ V ϕ ( S i + 1 ) − V ϕ ( S i ) {\displaystyle \delta _{i}=R_{i}+\gamma V_{\phi }(S_{i+1})-V_{\phi }(S_{i})} The critic parameters are updated by gradient descent on the squared TD error: ϕ ← ϕ − α ∇ ϕ ( δ i ) 2 = ϕ + α δ i ∇ ϕ V ϕ ( S i ) {\displaystyle \phi \leftarrow \phi -\alpha \nabla _{\phi }(\delta _{i})^{2}=\phi +\alpha \delta _{i}\nabla _{\phi }V_{\phi }(S_{i})} where α {\displaystyle \alpha } is the learning rate. Note that the gradient is taken with respect to the ϕ {\displaystyle \phi } in V ϕ ( S i ) {\displaystyle V_{\phi }(S_{i})} only, since the ϕ {\displaystyle \phi } in γ V ϕ ( S i + 1 ) {\displaystyle \gamma V_{\phi }(S_{i+1})} constitutes a moving target, and the gradient is not taken with respect to that. This is a common source of error in implementations that use automatic differentiation, and requires "stopping the gradient" at that point. Similarly, if the critic is estimating the action-value function Q π θ {\displaystyle Q^{\pi _{\theta }}} , then it can be learned by Q-learning or SARSA. In SARSA, the critic maintains an estimate of the Q-function, parameterized by ϕ {\displaystyle \phi } , denoted as Q ϕ ( s , a ) {\displaystyle Q_{\phi }(s,a)} . The temporal difference error is then calculated as δ i = R i + γ Q θ ( S i + 1 , A i + 1 ) − Q θ ( S i , A i ) {\displaystyle \delta _{i}=R_{i}+\gamma Q_{\theta }(S_{i+1},A_{i+1})-Q_{\theta }(S_{i},A_{i})} . The critic is then updated by θ ← θ + α δ i ∇ θ Q θ ( S i , A i ) {\displaystyle \theta \leftarrow \theta +\alpha \delta _{i}\nabla _{\theta }Q_{\theta }(S_{i},A_{i})} The advantage critic can be trained by training both a Q-function Q ϕ ( s , a ) {\displaystyle Q_{\phi }(s,a)} and a state-value function V ϕ ( s ) {\displaystyle V_{\phi }(s)} , then let A ϕ ( s , a ) = Q ϕ ( s , a ) − V ϕ ( s ) {\displaystyle A_{\phi }(s,a)=Q_{\phi }(s,a)-V_{\phi }(s)} . Although, it is more common to train just a state-value function V ϕ ( s ) {\displaystyle V_{\phi }(s)} , then estimate the advantage by A ϕ ( S i , A i ) ≈ ∑ j ∈ 0 : n − 1 γ j R i + j + γ n V ϕ ( S i + n ) − V ϕ ( S i ) {\displaystyle A_{\phi }(S_{i},A_{i})\approx \sum _{j\in 0:n-1}\gamma ^{j}R_{i+j}+\gamma ^{n}V_{\phi }(S_{i+n})-V_{\phi }(S_{i})} Here, n {\displaystyle n} is a positive integer. The higher n {\displaystyle n} is, the more lower is the bias in the advantage estimation, but at the price of higher variance. The Generalized Advantage Estimation (GAE) introduces a hyperparameter λ {\displaystyle \lambda } that smoothly interpolates between Monte Carlo returns ( λ = 1 {\displaystyle \lambda =1} , high variance, no bias) and 1-step TD learning ( λ = 0 {\displaystyle \lambda =0} , low variance, high bias). This hyperparameter can be adjusted to pick the optimal bias-variance trade-off in advantage estimation. It uses an exponentially decaying average of n-step returns with λ {\displaystyle \lambda } being the decay strength. == Variants == Asynchronous Advantage Actor-Critic (A3C): Parallel and asynchronous version of A2C. Soft Actor-Critic (SAC): Incorporates entropy maximization for improved exploration. Deep Deterministic Policy Gradient (DDPG): Specialized for continuous action spaces.</p> <h2><a href="https://aizhi.co/html/448f799544.html" title="ARMA International">ARMA International</a></h2> <p>ARMA International (formerly the Association of Records Managers and Administrators) is an American not-for-profit professional association for information professionals – primarily information management (including records management) and information governance, and related industry practitioners and vendors. The association provides educational opportunities and publications covering aspects of information management broadly. == History == The Association was founded in 1955. In 1975, the Association of Records Executives and Administrators (AREA) and the American Records Management Association merged to form ARMA International. The headquarters for ARMA International is located in Overland Park, Kansas. == Operations == ARMA International services professionals in the United States, Canada, Japan, and the United Kingdom. Its members include records managers, attorneys, information technology professionals, consultants, and archivists involved in various aspects of managing records and information assets. ARMA hosts an annual conference with the goal of bringing together record and information management professionals from around the world – In 2023, ARMA hosted conferences in both the United States and Canada. Topics addressed in the 120+ educational sessions include advanced technology, creating information structure, ediscovery and information law, information management fundamentals, information project management, and reducing organizational information risk. The expo features exhibitors displaying records and information technologies, products, and services.</p> </div> <nav class="article-pagination" aria-label="More guides"> <div class="prev-article"> <span>← Previous</span> <a href="https://aizhi.co/html/468a399528.html" title="Legendre moment">Legendre moment</a> </div> <div class="next-article"> <span>Next →</span> <a href="https://aizhi.co/html/128f299869.html" title="Round-trip engineering">Round-trip engineering</a> </div> </nav> </article> <section class="related-articles" aria-label="Related articles"> <h2>Related Articles</h2> <ul> <li> <a href="https://aizhi.co/html/426b199572.html" title="Voice search">Voice search</a> <time datetime="2026-06-03 02:39">2026-06-03 02:39</time> </li> <li> <a href="https://aizhi.co/html/482c799510.html" title="Information literacy">Information literacy</a> <time datetime="2026-06-03 02:12">2026-06-03 02:12</time> </li> <li> <a href="https://aizhi.co/news/287b799705.html" title="Automated journalism">Automated journalism</a> <time datetime="2026-06-03 01:36">2026-06-03 01:36</time> </li> <li> <a href="https://aizhi.co/news/481e799511.html" title="Information school">Information school</a> <time datetime="2026-06-03 01:05">2026-06-03 01:05</time> </li> <li> <a href="https://aizhi.co/html/394e399602.html" title="Neural field">Neural field</a> <time datetime="2026-06-03 00:26">2026-06-03 00:26</time> </li> <li> <a href="https://aizhi.co/news/00a799992.html" title="PureXML">PureXML</a> <time datetime="2026-06-03 00:22">2026-06-03 00:22</time> </li> </ul> </section> </main> <aside class="sidebar"> <section class="sidebar-section"> <h2>Categories</h2> <ul> <li><a href="https://aizhi.co/aiimagegenerators/">AI Image Generators</a></li><li><a href="https://aizhi.co/aicodingtools/">AI Coding Tools</a></li><li><a href="https://aizhi.co/ainewsandguides/">AI News and Guides</a></li><li><a href="https://aizhi.co/aiwritingtools/">AI Writing Tools</a></li><li><a href="https://aizhi.co/aichatbotsandassistants/">AI Chatbots and Assistants</a></li><li><a href="https://aizhi.co/aiforbusiness/">AI for Business</a></li><li><a href="https://aizhi.co/aivideotools/">AI Video Tools</a></li> </ul> </section> <section class="sidebar-section"> <h2>Latest Articles</h2> <ul> <li><a href="https://aizhi.co/html/241f099758.html" title="List of security assessment tools">List of security assessment tools</a></li><li><a href="https://aizhi.co/news/255d799737.html" title="Long division">Long division</a></li><li><a href="https://aizhi.co/news/375a799617.html" title="In-place algorithm">In-place algorithm</a></li><li><a href="https://aizhi.co/html/248c799744.html" title="Snap rounding">Snap rounding</a></li><li><a href="https://aizhi.co/news/106b299891.html" title="XLeratorDB">XLeratorDB</a></li><li><a href="https://aizhi.co/news/477b799515.html" title="Known-item search">Known-item search</a></li><li><a href="https://aizhi.co/news/455c799537.html" title="Two-phase commit protocol">Two-phase commit protocol</a></li><li><a href="https://aizhi.co/html/71b799921.html" title="Small Data">Small Data</a></li><li><a href="https://aizhi.co/news/25c499970.html" title="And–or tree">And–or tree</a></li><li><a href="https://aizhi.co/news/257c799735.html" title="Reservoir sampling">Reservoir sampling</a></li> </ul> </section> </aside> </div> </div> </div> <footer class="site-footer"> <div class="container"> <div class="footer-cols"> <div class="footer-col footer-about"> <a class="brand" href="https://aizhi.co/" aria-label="Aizhi"> <span class="brand-mark" aria-hidden="true">✦</span> <span class="brand-text">Aizhi</span> </a> <p class="footer-tagline">Hand-picked AI tools, generators and practical how-to guides — independent reviews, updated for 2026.</p> </div> <nav class="footer-col" aria-label="Categories"> <h2 class="footer-h">Categories</h2> <ul> <li><a href="https://aizhi.co/ainewsandguides/">AI News and Guides</a></li><li><a href="https://aizhi.co/aivideotools/">AI Video Tools</a></li><li><a href="https://aizhi.co/aicodingtools/">AI Coding Tools</a></li><li><a href="https://aizhi.co/aiimagegenerators/">AI Image Generators</a></li><li><a href="https://aizhi.co/aiwritingtools/">AI Writing Tools</a></li><li><a href="https://aizhi.co/aiforbusiness/">AI for Business</a></li><li><a href="https://aizhi.co/aichatbotsandassistants/">AI Chatbots and Assistants</a></li> </ul> </nav> <nav class="footer-col" aria-label="Site"> <h2 class="footer-h">Site</h2> <ul> <li><a href="https://aizhi.co/">Home</a></li> <li><a href="/sitemap.xml">XML Sitemap</a></li> </ul> </nav> </div> <div class="partner-links" aria-label="Network"> </div> <p class="footer-copy"> © Aizhi. All rights reserved. </p> </div> </footer> </body> </html>