In computing, external memory algorithms or out-of-core algorithms are algorithms that are designed to process data that are too large to fit into a computer's main memory at once. Such algorithms must be optimized to efficiently fetch and access data stored in slow bulk memory (auxiliary memory) such as hard drives or tape drives, or when memory is on a computer network. External memory algorithms are analyzed in the external memory model. == Model == External memory algorithms are analyzed in an idealized model of computation called the external memory model (or I/O model, or disk access model). The external memory model is an abstract machine similar to the RAM machine model, but with a cache in addition to main memory. The model captures the fact that read and write operations are much faster in a cache than in main memory, and that reading long contiguous blocks is faster than reading randomly using a disk read-and-write head. The running time of an algorithm in the external memory model is defined by the number of reads and writes to memory required. The model was introduced by Alok Aggarwal and Jeffrey Vitter in 1988. The external memory model is related to the cache-oblivious model, but algorithms in the external memory model may know both the block size and the cache size. For this reason, the model is sometimes referred to as the cache-aware model. The model consists of a processor with an internal memory or cache of size M, connected to an unbounded external memory. Both the internal and external memory are divided into blocks of size B. One input/output or memory transfer operation consists of moving a block of B contiguous elements from external to internal memory, and the running time of an algorithm is determined by the number of these input/output operations. == Algorithms == Algorithms in the external memory model take advantage of the fact that retrieving one object from external memory retrieves an entire block of size B. This property is sometimes referred to as locality. Searching for an element among N objects is possible in the external memory model using a B-tree with branching factor B. Using a B-tree, searching, insertion, and deletion can be achieved in O ( log B N ) {\displaystyle O(\log _{B}N)} time (in Big O notation). Information theoretically, this is the minimum running time possible for these operations, so using a B-tree is asymptotically optimal. External sorting is sorting in an external memory setting. External sorting can be done via distribution sort, which is similar to quicksort, or via a M B {\displaystyle {\tfrac {M}{B}}} -way merge sort. Both variants achieve the asymptotically optimal runtime of O ( N B log M B N B ) {\displaystyle O\left({\frac {N}{B}}\log _{\frac {M}{B}}{\frac {N}{B}}\right)} to sort N objects. This bound also applies to the fast Fourier transform in the external memory model. The permutation problem is to rearrange N elements into a specific permutation. This can either be done either by sorting, which requires the above sorting runtime, or inserting each element in order and ignoring the benefit of locality. Thus, permutation can be done in O ( min ( N , N B log M B N B ) ) {\displaystyle O\left(\min \left(N,{\frac {N}{B}}\log _{\frac {M}{B}}{\frac {N}{B}}\right)\right)} time. == Applications == The external memory model captures the memory hierarchy, which is not modeled in other common models used in analyzing data structures, such as the random-access machine, and is useful for proving lower bounds for data structures. The model is also useful for analyzing algorithms that work on datasets too big to fit in internal memory. A typical example is geographic information systems, especially digital elevation models, where the full data set easily exceeds several gigabytes or even terabytes of data. This methodology extends beyond general purpose CPUs and also includes GPU computing as well as classical digital signal processing. In general-purpose computing on graphics processing units (GPGPU), powerful graphics cards (GPUs) with little memory (compared with the more familiar system memory, which is most often referred to simply as RAM) are utilized with relatively slow CPU-to-GPU memory transfer (when compared with computation bandwidth). == History == An early use of the term "out-of-core" as an adjective is in 1962 in reference to devices that are other than the core memory of an IBM 360. An early use of the term "out-of-core" with respect to algorithms appears in 1971.
Yorba (software)
Yorba is a web-based personal information management platform for finding, monitoring, or deleting online accounts and subscriptions. Yorba is a participating member of Consumer Reports’ Data Rights Protocol (DRP) consortium that develops open technical standards for exercising consumer data rights under laws including the California Consumer Privacy Act. == History == Yorba began as a research project around 2021. It was founded by Chris Zeunstrom (CEO), Nolan Cabeje (CDO) and David Schmudde (CTO). Zeunstrom says he began developing Yorba after growing frustrated with managing numerous email accounts, noting overloaded inboxes create distraction and potential security vulnerabilities. Yorba’s early development was also influenced by security issues he encountered at a previous company, which had been affected by data breaches at a time when such incidents were becoming increasingly common. In 2023, Yorba launched a private beta as a public benefit corporation funded through a give-back model operated by Zeunstrom's New York-based design firm, Ruca. In January 2024, Yorba entered public beta and reported over 1,000 users, including 160 premium subscribers. At the time of the public beta launch, Yorba integrated with Gmail and announced plans to expand compatibility to other online services and cloud storage providers. In September 2024, Yorba completed conformance testing under the Data Rights Protocol, an initiative developed by Consumer Reports, to establish a standard and open-source framework for securely transmitting consumer data rights requests under laws like the California Consumer Privacy Act. Yorba was named among twelve participating companies that implemented the protocol alongside OneTrust and Consumer Reports’ own Permission Slip app. Yorba was one of nine startups selected as 2025 finalist in the Santander X Global Awards international entrepreneurship competition. == Features == Yorba scans user inbox history data to identify online accounts, mailing lists, and possible data breaches. It uses natural language processing and machine learning to identify a user's accounts, services, and subscriptions. The platform prompts password resets for compromised accounts and locates unused accounts. The platform also supports mailing list management by identifying and helping users unsubscribe from newsletters. Paid subscribers can locate and cancel recurring charges. Yorba links with financial institutions in the U.S., Canada, and EU via Plaid Inc. to detect recurring charges and delete unwanted subscriptions. == Privacy and Ethics == Yorba's founder has openly criticized dark patterns that make canceling services difficult, citing personal frustration with inbox clutter as part of his inspiration for Yorba. Yorba offers privacy policy analysis in partnership with Amsterdam-based nonprofit Terms of Service; Didn’t Read, assigning grades based on invasiveness or ethical concerns. As of 2024, the company described its pricing as designed to cover operational costs and sustain the platform without outside investment.
Contrast set learning
Contrast set learning is a form of association rule learning that seeks to identify meaningful differences between separate groups by reverse-engineering the key predictors that identify for each particular group. For example, given a set of attributes for a pool of students (labeled by degree type), a contrast set learner would identify the contrasting features between students seeking bachelor's degrees and those working toward PhD degrees. == Overview == A common practice in data mining is to classify, to look at the attributes of an object or situation and make a guess at what category the observed item belongs to. As new evidence is examined (typically by feeding a training set to a learning algorithm), these guesses are refined and improved. Contrast set learning works in the opposite direction. While classifiers read a collection of data and collect information that is used to place new data into a series of discrete categories, contrast set learning takes the category that an item belongs to and attempts to reverse engineer the statistical evidence that identifies an item as a member of a class. That is, contrast set learners seek rules associating attribute values with changes to the class distribution. They seek to identify the key predictors that contrast one classification from another. For example, an aerospace engineer might record data on test launches of a new rocket. Measurements would be taken at regular intervals throughout the launch, noting factors such as the trajectory of the rocket, operating temperatures, external pressures, and so on. If the rocket launch fails after a number of successful tests, the engineer could use contrast set learning to distinguish between the successful and failed tests. A contrast set learner will produce a set of association rules that, when applied, will indicate the key predictors of each failed tests versus the successful ones (the temperature was too high, the wind pressure was too high, etc.). Contrast set learning is a form of association rule learning. Association rule learners typically offer rules linking attributes commonly occurring together in a training set (for instance, people who are enrolled in four-year programs and take a full course load tend to also live near campus). Instead of finding rules that describe the current situation, contrast set learners seek rules that differ meaningfully in their distribution across groups (and thus, can be used as predictors for those groups). For example, a contrast set learner could ask, “What are the key identifiers of a person with a bachelor's degree or a person with a PhD, and how do people with PhD's and bachelor’s degrees differ?” Standard classifier algorithms, such as C4.5, have no concept of class importance (that is, they do not know if a class is "good" or "bad"). Such learners cannot bias or filter their predictions towards certain desired classes. As the goal of contrast set learning is to discover meaningful differences between groups, it is useful to be able to target the learned rules towards certain classifications. Several contrast set learners, such as MINWAL or the family of TAR algorithms, assign weights to each class in order to focus the learned theories toward outcomes that are of interest to a particular audience. Thus, contrast set learning can be thought of as a form of weighted class learning. === Example: Supermarket Purchases === The differences between standard classification, association rule learning, and contrast set learning can be illustrated with a simple supermarket metaphor. In the following small dataset, each row is a supermarket transaction and each "1" indicates that the item was purchased (a "0" indicates that the item was not purchased): Given this data, Association rule learning may discover that customers that buy onions and potatoes together are likely to also purchase hamburger meat. Classification may discover that customers that bought onions, potatoes, and hamburger meats were purchasing items for a cookout. Contrast set learning may discover that the major difference between customers shopping for a cookout and those shopping for an anniversary dinner are that customers acquiring items for a cookout purchase onions, potatoes, and hamburger meat (and do not purchase foie gras or champagne). == Treatment learning == Treatment learning is a form of weighted contrast-set learning that takes a single desirable group and contrasts it against the remaining undesirable groups (the level of desirability is represented by weighted classes). The resulting "treatment" suggests a set of rules that, when applied, will lead to the desired outcome. Treatment learning differs from standard contrast set learning through the following constraints: Rather than seeking the differences between all groups, treatment learning specifies a particular group to focus on, applies a weight to this desired grouping, and lumps the remaining groups into one "undesired" category. Treatment learning has a stated focus on minimal theories. In practice, treatment are limited to a maximum of four constraints (i.e., rather than stating all of the reasons that a rocket differs from a skateboard, a treatment learner will state one to four major differences that predict for rockets at a high level of statistical significance). This focus on simplicity is an important goal for treatment learners. Treatment learning seeks the smallest change that has the greatest impact on the class distribution. Conceptually, treatment learners explore all possible subsets of the range of values for all attributes. Such a search is often infeasible in practice, so treatment learning often focuses instead on quickly pruning and ignoring attribute ranges that, when applied, lead to a class distribution where the desired class is in the minority. === Example: Boston housing data === The following example demonstrates the output of the treatment learner TAR3 on a dataset of housing data from the city of Boston (a nontrivial public dataset with over 500 examples). In this dataset, a number of factors are collected for each house, and each house is classified according to its quality (low, medium-low, medium-high, and high). The desired class is set to "high", and all other classes are lumped together as undesirable. The output of the treatment learner is as follows: Baseline class distribution: low: 29% medlow: 29% medhigh: 21% high: 21% Suggested Treatment: [PTRATIO=[12.6..16), RM=[6.7..9.78)] New class distribution: low: 0% medlow: 0% medhigh: 3% high: 97% With no applied treatments (rules), the desired class represents only 21% of the class distribution. However, if one filters the data set for houses with 6.7 to 9.78 rooms and a neighborhood parent-teacher ratio of 12.6 to 16, then 97% of the remaining examples fall into the desired class (high-quality houses). == Algorithms == There are a number of algorithms that perform contrast set learning. The following subsections describe two examples. === STUCCO === The STUCCO contrast set learner treats the task of learning from contrast sets as a tree search problem where the root node of the tree is an empty contrast set. Children are added by specializing the set with additional items picked through a canonical ordering of attributes (to avoid visiting the same nodes twice). Children are formed by appending terms that follow all existing terms in a given ordering. The formed tree is searched in a breadth-first manner. Given the nodes at each level, the dataset is scanned and the support is counted for each group. Each node is then examined to determine if it is significant and large, if it should be pruned, and if new children should be generated. After all significant contrast sets are located, a post-processor selects a subset to show to the user - the low order, simpler results are shown first, followed by the higher order results which are "surprising and significantly different." The support calculation comes from testing a null hypothesis that the contrast set support is equal across all groups (i.e., that contrast set support is independent of group membership). The support count for each group is a frequency value that can be analyzed in a contingency table where each row represents the truth value of the contrast set and each column variable indicates the group membership frequency. If there is a difference in proportions between the contrast set frequencies and those of the null hypothesis, the algorithm must then determine if the differences in proportions represent a relation between variables or if it can be attributed to random causes. This can be determined through a chi-square test comparing the observed frequency count to the expected count. Nodes are pruned from the tree when all specializations of the node can never lead to a significant and large contrast set. The decision to prune is based on: The minimum deviation size: The maximum difference between the support
Strong secrecy
Strong secrecy is a term used in formal proof-based cryptography for making propositions about the security of cryptographic protocols. It is a stronger notion of security than syntactic (or weak) secrecy. Strong secrecy is related with the concept of semantic security or indistinguishability used in the computational proof-based approach. Bruno Blanchet provides the following definition for strong secrecy: Strong secrecy means that an adversary cannot see any difference when the value of the secret changes For example, if a process encrypts a message m an attacker can differentiate between different messages, since their ciphertexts will be different. Thus m is not a strong secret. If however, probabilistic encryption were used, m would be a strong secret. The randomness incorporated into the encryption algorithm will yield different ciphertexts for the same value of m.
Point-to-point encryption
Point-to-point encryption (P2PE) is a standard established by the PCI Security Standards Council. Payment solutions that offer similar encryption but do not meet the P2PE standard are referred to as end-to-end encryption (E2EE) solutions. The objective of P2PE and E2EE is to provide a payment security solution that instantaneously converts confidential payment card (credit and debit card) data and information into indecipherable code at the time the card is swiped, in order to prevent hacking and fraud. It is designed to maximize the security of payment card transactions in an increasingly complex regulatory environment. == The standard == The P2PE Standard defines the requirements that a "solution" must meet in order to be accepted as a PCI-validated P2PE solution. A "solution" is a complete set of hardware, software, gateway, decryption, device handling, etc. Only "solutions" can be validated; individual pieces of hardware such as card readers cannot be validated. It is also a common mistake to refer to P2PE validated solutions as "certified"; there is no such certification. The determination of whether or not a solution meets the P2PE standard is the responsibility of a P2PE Qualified Security Assessor (P2PE-QSA). P2PE-QSA companies are independent third-party companies who employ assessors that have met the PCI Security Standards Council's requirements for education and experience, and have passed the requisite exam. The PCI Security Standards Council does not validate solutions. == How it works == As a payment card is swiped through a card reading device, referred to as a point of interaction (POI) device, at the merchant location or point of sale, the device immediately encrypts the card information. A device that is part of a PCI-validated P2PE solution uses an algorithmic calculation to encrypt the confidential payment card data. From the POI, the encrypted, indecipherable codes are sent to the payment gateway or processor for decryption. The keys for encryption and decryption are never available to the merchant, making card data entirely invisible to the retailer. Once the encrypted codes are within the secure data zone of the payment processor, the codes are decrypted to the original card numbers and then passed to the issuing bank for authorization. The bank either approves or rejects the transaction, depending upon the card holder's payment account status. The merchant is then notified if the payment is accepted or rejected to complete the process along with a token that the merchant can store. This token is a unique number reference to the original transaction that the merchant can use should they ever be needed to perform research or refund the customer without ever knowing the customer's card information (tokenization). There are also Qualified Integrator and Reseller (QIR) Companies, which are businesses authorized to "implement, configure, and/or support validated" PA-DSS Payment Applications, and perform qualified installations. == Solution providers == According to the PCI Security Standards Council:The P2PE solution provider is a third-party entity (for example, a processor, acquirer, or payment gateway) that has overall responsibility for the design and implementation of a specific P2PE solution, and manages P2PE solutions for its merchant customers. The solution provider has overall responsibility for ensuring that all P2PE requirements are met, including any P2PE requirements performed by third-party organizations on behalf of the solution provider (for example, certification authorities and key-injection facilities). == Benefits == === Customer benefits === P2PE significantly reduces the risk of payment card fraud by instantaneously encrypting confidential cardholder data at the moment a payment card is swiped or "dipped" if it is a chip card at the card reading device (payment terminal) or POI. === Merchant benefits === P2PE significantly facilitates merchant responsibilities: With a P2PE validated solution, merchants save significant time and money as PCI requirements may be greatly reduced. Payment Card Industry Data Security Standard (PCI DSS). For organizations who use a P2PE validated solution provider, the PCI Self Assessment Questionnaire is reduced from 12 sections to 4 sections and the controls are reduced from 329 questions to just 35. In the event of fraud, the P2PE Solution Provider, not the merchant, is held accountable for data loss and resulting fines that may be assessed by the card brands (American Express, Visa, MasterCard, Discover, and JCB). The PCI Security Standards Council does not assess penalties on Solution Providers or Merchants. The payment process with P2PE is quicker than other transaction processes, thus creating simpler and faster customer–merchant transactions. == Point-to-point encryption versus end-to-end encryption == === Point-to-point === A point-to-point connection directly links system 1 (the point of payment card acceptance) to system 2 (the point of payment processing). A true P2PE solution is determined with three main factors: The solution uses a hardware-to-hardware encryption and decryption process along with a POI device that has SRED (Secure Reading and Exchange of Data) listed as a function. The solution has been validated to the PCI P2PE Standard which includes specific POI device requirements such as strict controls regarding shipping, receiving, tamper-evident packaging, and installation. A solution includes merchant education in the form of a P2PE Instruction Manual, which guides the merchant on POI device use, storage, return for repairs, and regular PCI reporting. === End-to-end === End-to-end encryption as the name suggests has the advantage over P2PE that card details are not unencrypted between the two endpoints. If the endpoints are a PCI PED validated PIN pad and a POS acquirer, there is no opportunity for the card details to be intercepted. It is obviously important that the endpoints (the PED and gateway) are provided by PCI accredited organisations. == PCI point-to-point encryption requirements == The requirements include: Secure encryption of payment card data at the point of interaction (POI), P2PE validated application(s) at the point of interaction, Secure management of encryption and decryption devices, Management of the decryption environment and all decrypted account data, Use of secure encryption methodologies and cryptographic key operations, including key generation, distribution, loading/injection, administration, and usage.
WomanStats Project
The WomanStats Project is a donor-funded research and database project housed at Brigham Young University that "seeks to collect detailed statistical data on the status of women around the world, and to connect that data with data on the security of states." The WomanStats Database aims to provide a comprehensive compilation of information on the status of women in the world. Coders comb the extant literature and conduct expert interviews to find qualitative and quantitative information on over 300 indicators of women's status in 174 countries with populations of at least 200,000. Access to the online database is free. == History and structure == WomanStats began as an outgrowth of a paper Dr. Valerie M. Hudson (of the Brigham Young University Political Science department) and one of her graduate students, Andrea den Boer, published in International Security on the association between national security and the abnormal sex ratio in Asia. After the success and influence of their first article, (later added as one of their top twenty national security articles of that journal of all time), Hudson and den Boer did further research on the connection between the status of women and national security, but found that there was no single database that covered the range of topics that they needed for their research. Consequently, they began compiling information on variables regarding the status of women around the world. The database was officially formed in 2001 and grew exponentially as it later added more variables. The Project went live on the Internet in July 2007. The principal investigators are: Valerie M. Hudson (International Relations), Bonnie Ballif-Spanvill (Psychology, emeritus), and Chad F. Emmett (Geography) all from Brigham Young University, Mary Caprioli from the University of Minnesota, Duluth (International Relations), Rose McDermott from Brown University (International Relations), Andrea Den Boer from the University of Kent at Canterbury in the United Kingdom (International Relations) and S. Matthew Stearmer from the Ohio State University (Sociology; doctoral student). Approximately a dozen undergraduate and graduate students at Brigham Young University and Texas A&M University work at any one time as coders for the project. The coders take the raw quantitative and qualitative data collected in government reports, news articles, research papers, etc. and sort the applicable information on women into categories. They may also implement scales developed by the principal investigators, or that they (the students) themselves have developed. == Database == As of February 2011, the database has 307 variables, covers 174 nations with populations over 200,000, uses 18,015 sources and contains over 111,000 individual data points. All data is referenced to original sources. Not every variable has information for each country; similarly, not all countries have information for each variable: overall, about 70% of country-variable combinations have information. These database coding gaps exist where information is not available or is incomplete, or variables are not collected and reported by governments or international organizations. At times, information from different sources may be contradictory, and the WomanStats Database records this discrepant information for triangulation purposes. == Users and role of the database == The database is meant to help fill a hole in the extant data on the situation of women around the world. WomanStats data and research has been vetted and/or used by the United Nations, the United States Department of Defense, the Central Intelligence Agency, and the World Bank. Their data and research were also used by the United States Senate Committee on Foreign Relations in crafting the International Violence Against Women’s Act. The Inter-Agency Network on Women and Gender Equality (IANWGE) of the United Nations has stated that the WomanStats project "filled a major gap in the availability of data on women" (2007). Victor Asal and Mitchell Brown, researchers not affiliated with WomanStats, stated in an article published in Politics and Policy that "one of the most significant challenges of cross-national empirical studies of the prevalence of interpersonal violence is the paucity of available data, particularly reliable data," and that "WomanStats has allowed for an important first glimpse at analyzing the factors related to interpersonal violence." They conclude by stating that "Our findings suggest that, in the same way that larger disciplinary resources have invested in interstate and intrastate war, disciplinary resources need to be expended in creating a data set exploring interpersonal violence. Until the rights and the lives of women and children are taken as seriously as the survival of states by more proactively collaborating on projects like WomanStats, we will continue to only have a small lens through which to understand problems like this." Princeton University professor Evan S. Liberman wrote, "Although data on political regimes and group conflict have been in far greater demand by political scientists than data on gender politics and policies, two gender-related databases provide...examples of innovative HIRDs. Both the Womanstats database project (Hudson et al. 2009) and the Research Network on Gender Politics and the State (RNGS) project (McBride et al. 2008) are well-integrated presentations of quantitative and qualitative data characterizing the quality of gender relations around the world and, in particular, analytic descriptions of the treatment of women."." == Research == The research component of WomanStats focuses on exploring the relationship between the situation of women and the behavior and security of states. Current research initiatives include: Exploring the relationship between violent instability and inequity and family law. Examining the effect of polygyny and marriage market dislocations on the rise of suicide terrorism. Documenting discrepancies between laws on the books and cultural practices on the ground concerning gender issues. Investigating how well the situation of women predicts the peacefulness of nations-states, compared to their variables such as democracy, wealth, and civilization. The Project has published articles in International Security, International Studies Quarterly, Peace and Conflict, Journal of Peace Research, Political Psychology, Cumberland Law Review, and World Political Review, and has a forthcoming book from Columbia University Press.
Client-side encryption
Client-side encryption is the cryptographic technique of encrypting data on the sender's side, before it is transmitted to a server such as a cloud storage service. Client-side encryption features an encryption key that is not available to the service provider, making it difficult or impossible for service providers to decrypt hosted data. Client-side encryption allows for the creation of applications whose providers cannot access the data its users have stored, thus offering a high level of privacy. Applications utilizing client-side encryption are sometimes marketed under the misleading or incorrect term "zero-knowledge", but this is a misnomer, as the term zero-knowledge describes something entirely different in the context of cryptography. == Details == Client-side encryption seeks to eliminate the potential for data to be viewed by service providers (or third parties that compel service providers to deliver access to data), client-side encryption ensures that data and files that are stored in the cloud can only be viewed on the client-side of the exchange. This prevents data loss and the unauthorized disclosure of private or personal files, providing increased peace of mind for its users. Current recommendations by industry professionals as well as academic scholars offer great vocal support for developers to include client-side encryption to protect the confidentiality and integrity of information. === Examples of services that use client-side encryption by default === Tresorit MEGA Cryptee Cryptomator === Examples of services that optionally support client-side encryption === Apple iCloud offers optional client-side encryption when "Advanced Data Protection for iCloud" is enabled. Google Drive, Google Docs, Google Meet, Google Calendar, and Gmail — However, as of Jul 2024, optional client-side encryption features are only available to paid users. === Examples of services that do not support client-side encryption === Dropbox === Examples of client-side encrypted services that no longer exist === SpiderOak Backup