Friending and following

Friending and following

Friending is the act of adding someone to a list of "friends" on a social networking service. The notion does not necessarily involve the concept of friendship. It is also distinct from the idea of a "fan"—as employed on the WWW sites of businesses, bands, artists, and others—since it is more than a one-way relationship. A "fan" only receives things. A "friend" can communicate back to the person friending. The act of "friending" someone usually grants that person special privileges (on the service) with respect to oneself. On Facebook, for example, one's "friends" have the privilege of viewing and posting to one's "timeline". Following is a similar concept on other social network services, such as Twitter and Instagram, where a person (follower) chooses to add content from a person or page to their newsfeed. Unlike friending, following is not necessarily mutual, and a person can unfollow (stop following) or block another user at any time without affecting that user's following status. The first scholarly definition and examination of friending and defriending (the act of removing someone from one's friend list, also called unfriending) was David Fono and Kate Raynes-Goldie's "Hyperfriendship and beyond: Friends and Social Norms on LiveJournal" from 2005, which identified the use of the term as both a noun and a verb by users of early social network site and blogging platform LiveJournal, which was originally launched in 1999. == Friend/follower count, friend collecting, and multiple accounts == The addition of people to a friend list without regard to whether one actually is their friend is sometimes known as friend whoring. Matt Jones of Dopplr went so far as to coin the expression "friending considered harmful" to describe the problem of focusing upon the friending of more and more people at the expense of actually making any use of a social network. Friend collecting is the adding of hundreds or thousands of friends/followers, a not uncommon order of magnitude on some social sites. As a result, many teen users feel pressured to heavily curate their posts, posting only carefully posed and edited photographs with well-thought-out captions. Some Instagram users will create a second account, known as a Finsta (short for "Fake Instagram"). A Finsta is typically private, and the owner only allows close friends to follow it. Since the follower count is kept down, the posts can be more candid and silly in nature. Users may also create multiple accounts based on their interests. Someone with a personal social media account might be a photographer and maintain a separate account for that. There is risk associated with following large numbers of people: scholars say that social anxiety could be an effect of managing a large social media network, as users can feel jealous and have a "fear of missing out". == Unfriending and unfollowing == Unfriending is the act of removing someone from a friends list. On Facebook, this means the action is unilateral, meaning, the friendship is terminated on both sides. The act of unfriending is often used when one user was flirting and made the other uncomfortable. Unfollowing is a little different. When a user unfollows someone on Instagram or Twitter, it continues a one-sided relationship. Often, the unfollowed user doesn't realize they were unfollowed, so they continue the following. == Social network friending and friendship == There are distinct groups of "friends" that one can friend on a social networking service. The notion of a social network friend does not necessarily embody the concept of friendship. Although terminology has not yet evolved to distinguish the different types of social networking friends, they can be broken into the following three categories. friends who are actually known These are people that may be one's friends or family in real life, with whom one has regular interaction either on-line or off-line. organizational friends These are companies and other organizations who maintain a "friending" relationship as a contacts list. complete strangers These are social networking "friends" with whom one has no relationship at all. Within these categories "friends" can be made up of strong ties, weak existing ties, weak latent ties, and parasocial ties. Strong ties can be made up of close family members and friends where self-disclosure, intimacy and frequent content occur. Weak existing ties can be made up of acquaintances, co-workers and distance relatives with whom the user has inconsistent contact. Weak latent ties can be made up of people within a similar geographical location or profession that can be used as a potential future bridge to other connections. Parasocial ties can be made up of celebrities, public figures and media personas. Human nature is to reciprocate a friending, marking someone as a friend who has marked oneself as a friend. This is a social norm for social networking services. However, this leads to mixing up who is an actual friend, and who is a contact. Tagging someone as a "contact" who has marked one as a "friend" can be perceived as impolite. Other concerns about this issue are treated in Sherry Turkle's Alone Together which analyses many behavioral dynamics in social media friendships. Turkle defines herself as "cautiously optimistic", but expresses concern that distance communications may undermine genuine face-to-face spoken discourses, lessening people's expectations of one another. One social networking service, FriendFeed, allows one to friend someone as a "fake" friend. The person "fake" friended receives the usual notifications for friending, but that person's updates are not received. Gavin Bell, author of Building Social Web Applications, describes this mechanism as "ludicrous". Results from a 2007 survey the Center for the Digital Future stated that only 23% of internet users have at least one virtual friend whom they have only met online. Ideally the number of virtual friends is directly proportional to the use of the Internet, but the same survey showed 20% of heavy-users (more than 3 hours/day) who claimed an average of 8.7% online friends, reported at least one relationship that started virtually and migrated to in-person contact. This results and other concerning issues are included in the book Networked: The New Social Operating System co-written by Lee Rainie and Barry Wellman in 2012. == Ethical considerations == The act of "friending" someone on a social networking service has particular ethical implications for judges in the United States. Judicial codes of conducts in the various states generally incorporate some form of provision that judges should avoid even the appearance of impropriety. Whether this regulates and even prohibits judges "friending" attorneys that appear before them, and law enforcement personnel, has been the subject of some analysis by the judicial ethics panels of the various states. They haven't all agreed on the guidance that they have given to judges: The New York state Judicial Ethics committee in 2009 simply advised judges to employ caution, noting that the issue of "friending" someone on a social networking service is a publicly observable act that has little difference from other public behavior concerns judges already face. The Florida Judicial Ethics Advisory committee in 2009 noted that, judges being normal human beings, it was unavoidable for judges to form friendships without the responsibilities of their job. It prohibited judges from friending any attorneys that appeared before them, whilst allowing friending of those who do not, on the grounds that it may give the appearance to the general public (even if the substance is otherwise) that those attorneys who are friended hold special sway with the judge. A minority opinion of the committee asserted that there is a substantive difference between "friending" on a social networking service and actual friendship, and that the general public, being aware of the norms of social networking services, was capable of drawing this distinction and would not reasonably conclude either a special degree of influence or a violation of the code of judicial conduct. This minority opinion was outnumbered twice in 2009, both in the Judicial Ethics Advisory and in the Florida Supreme Court Judicial Ethics Advisory committee. The South Carolina judicial conduct committee in 2009 permitted judges to friend attorneys and law enforcement personnel, with the proviso that no judicial business should be conducted upon nor discussed via the social networking service. "... a judge should not become isolated from the community in which the judge lives.", the committee stated. The Kentucky Judicial Ethics committee in 2010 took the same position as the minority opinion in Florida. It urged judges to exercise caution, but recognized that the act of friending "does not, in and of itself, indicate the degree or intensity of a judge's relationship with the person who is the 'friend'

Seq2seq

Seq2seq is a family of machine learning approaches used for natural language processing. Originally developed by Lê Viết Quốc, a Vietnamese computer scientist and a machine learning pioneer at Google Brain, this framework has become foundational in many modern AI systems. Applications include language translation, image captioning, conversational models, speech recognition, and text summarization. Seq2seq uses sequence transformation: it turns one sequence into another sequence. == History == One naturally wonders if the problem of translation could conceivably be treated as a problem in cryptography. When I look at an article in Russian, I say: 'This is really written in English, but it has been coded in some strange symbols. I will now proceed to decode. seq2seq is an approach to machine translation (or more generally, sequence transduction) with roots in information theory, where communication is understood as an encode-transmit-decode process, and machine translation can be studied as a special case of communication. This viewpoint was elaborated, for example, in the noisy channel model of machine translation. In practice, seq2seq maps an input sequence into a real-numerical vector by using a neural network (the encoder), and then maps it back to an output sequence using another neural network (the decoder). The idea of encoder-decoder sequence transduction had been developed in the early 2010s. The papers most commonly cited as the originators that produced seq2seq are two papers from 2014. In the seq2seq as proposed by them, both the encoder and the decoder were LSTMs. This had the "bottleneck" problem, since the encoding vector has a fixed size, so for long input sequences, information would tend to be lost, as they are difficult to fit into the fixed-length encoding vector. The attention mechanism, proposed in 2014, resolved the bottleneck problem. They called their model RNNsearch, as it "emulates searching through a source sentence during decoding a translation". A problem with seq2seq models at this point was that recurrent neural networks are difficult to parallelize. The 2017 publication of Transformers resolved the problem by replacing the encoding RNN with self-attention Transformer blocks ("encoder blocks"), and the decoding RNN with cross-attention causally-masked Transformer blocks ("decoder blocks"). === Priority dispute === One of the papers cited as the originator for seq2seq is (Sutskever et al 2014), published at Google Brain while they were on Google's machine translation project. The research allowed Google to overhaul Google Translate into Google Neural Machine Translation in 2016. Tomáš Mikolov claims to have developed the idea (before joining Google Brain) of using a "neural language model on pairs of sentences... and then [generating] translation after seeing the first sentence"—which he equates with seq2seq machine translation, and to have mentioned the idea to Ilya Sutskever and Quoc Le (while at Google Brain), who failed to acknowledge him in their paper. Mikolov had worked on RNNLM (using RNN for language modelling) for his PhD thesis, and is more notable for developing word2vec. == Architecture == The main reference for this section is. === Encoder === The encoder is responsible for processing the input sequence and capturing its essential information, which is stored as the hidden state of the network and, in a model with attention mechanism, a context vector. The context vector is the weighted sum of the input hidden states and is generated for every time instance in the output sequences. === Decoder === The decoder takes the context vector and hidden states from the encoder and generates the final output sequence. The decoder operates in an autoregressive manner, producing one element of the output sequence at a time. At each step, it considers the previously generated elements, the context vector, and the input sequence information to make predictions for the next element in the output sequence. Specifically, in a model with attention mechanism, the context vector and the hidden state are concatenated together to form an attention hidden vector, which is used as an input for the decoder. The seq2seq method developed in the early 2010s uses two neural networks: an encoder network converts an input sentence into numerical vectors, and a decoder network converts those vectors to sentences in the target language. The Attention mechanism was grafted onto this structure in 2014 and is shown below. Later it was refined into the encoder-decoder Transformer architecture of 2017. === Training vs prediction === There is a subtle difference between training and prediction. During training time, both the input and the output sequences are known. During prediction time, only the input sequence is known, and the output sequence must be decoded by the network itself. Specifically, consider an input sequence x 1 : n {\displaystyle x_{1:n}} and output sequence y 1 : m {\displaystyle y_{1:m}} . The encoder would process the input x 1 : n {\displaystyle x_{1:n}} step by step. After that, the decoder would take the output from the encoder, as well as the as input, and produce a prediction y ^ 1 {\displaystyle {\hat {y}}_{1}} . Now, the question is: what should be input to the decoder in the next step? A standard method for training is "teacher forcing". In teacher forcing, no matter what is output by the decoder, the next input to the decoder is always the reference. That is, even if y ^ 1 ≠ y 1 {\displaystyle {\hat {y}}_{1}\neq y_{1}} , the next input to the decoder is still y 1 {\displaystyle y_{1}} , and so on. During prediction time, the "teacher" y 1 : m {\displaystyle y_{1:m}} would be unavailable. Therefore, the input to the decoder must be y ^ 1 {\displaystyle {\hat {y}}_{1}} , then y ^ 2 {\displaystyle {\hat {y}}_{2}} , and so on. It is found that if a model is trained purely by teacher forcing, its performance would degrade during prediction time, since generation based on the model's own output is different from generation based on the teacher's output. This is called exposure bias or a train/test distribution shift. A 2015 paper recommends that, during training, randomly switch between teacher forcing and no teacher forcing. === Attention for seq2seq === The attention mechanism is an enhancement introduced by Bahdanau et al. in 2014 to address limitations in the basic Seq2Seq architecture where a longer input sequence results in the hidden state output of the encoder becoming irrelevant for the decoder. It enables the model to selectively focus on different parts of the input sequence during the decoding process. At each decoder step, an alignment model calculates the attention score using the current decoder state and all of the attention hidden vectors as input. An alignment model is another neural network model that is trained jointly with the seq2seq model used to calculate how well an input, represented by the hidden state, matches with the previous output, represented by attention hidden state. A softmax function is then applied to the attention score to get the attention weight. In some models, the encoder states are directly fed into an activation function, removing the need for alignment model. An activation function receives one decoder state and one encoder state and returns a scalar value of their relevance. Consider the seq2seq language English-to-French translation task. To be concrete, let us consider the translation of "the zone of international control ", which should translate to "la zone de contrôle international ". Here, we use the special token as a control character to delimit the end of input for both the encoder and the decoder. An input sequence of text x 0 , x 1 , … {\displaystyle x_{0},x_{1},\dots } is processed by a neural network (which can be an LSTM, a Transformer encoder, or some other network) into a sequence of real-valued vectors h 0 , h 1 , … {\displaystyle h_{0},h_{1},\dots } , where h {\displaystyle h} stands for "hidden vector". After the encoder has finished processing, the decoder starts operating over the hidden vectors, to produce an output sequence y 0 , y 1 , … {\displaystyle y_{0},y_{1},\dots } , autoregressively. That is, it always takes as input both the hidden vectors produced by the encoder, and what the decoder itself has produced before, to produce the next output word: ( h 0 , h 1 , … {\displaystyle h_{0},h_{1},\dots } , "") → "la" ( h 0 , h 1 , … {\displaystyle h_{0},h_{1},\dots } , " la") → "la zone" ( h 0 , h 1 , … {\displaystyle h_{0},h_{1},\dots } , " la zone") → "la zone de" ... ( h 0 , h 1 , … {\displaystyle h_{0},h_{1},\dots } , " la zone de contrôle international") → "la zone de contrôle international " Here, we use the special token as a control character to delimit the start of input for the decoder. The decoding terminates as soon as "" appears in the decoder output. ==

Database-as-IPC

In computer programming, Database-as-IPC may be considered an anti-pattern where a disk persisted table in a database is used as the message queue store for routine inter-process communication (IPC) or subscribed data processing. If database performance is of concern, alternatives include sockets, network socket, or message queue. British computer scientist, Junade Ali, defined the Database-as-IPC Anti-Pattern as using a database to "schedule jobs or queue up tasks to be completed", noting that this anti-pattern centres around using a database for temporary messages instead of persistent data. == Controversy == The issue arises if there is a performance issue, and if additional systems (and servers) can be justified. In terms of performance, recent advancements in database systems provide more efficient mechanisms for signaling and messaging, and database systems also support memory (non-persisted) tables. There are databases with built-in notification mechanisms, such as PostgreSQL, SQL Server, and Oracle. These mechanisms and future improvements of database systems can make queuing much more efficient and avoid the need to set up a separate signaling or messaging queue system along with the server and management overhead. While MySQL doesn't have direct support for notifications, some workarounds are possible. However, they would be seen as non-standard and therefore more difficult to maintain.

Event condition action

Event condition action (ECA) is a short-cut for referring to the structure of active rules in event-driven architecture and active database systems. Such a rule traditionally consisted of three parts: The event part specifies the signal that triggers the invocation of the rule The condition part is a logical test that, if satisfied or evaluates to true, causes the action to be carried out The action part consists of updates or invocations on the local data This structure was used by the early research in active databases which started to use the term ECA. Current state of the art ECA rule engines use many variations on rule structure. Also other features not considered by the early research is introduced, such as strategies for event selection into the event part. In a memory-based rule engine, the condition could be some tests on local data and actions could be updates to object attributes. In a database system, the condition could simply be a query to the database, with the result set (if not null) being passed to the action part for changes to the database. In either case, actions could also be calls to external programs or remote procedures. Note that for database usage, updates to the database are regarded as internal events. As a consequence, the execution of the action part of an active rule can match the event part of the same or another active rule, thus triggering it. The equivalent in a memory-based rule engine would be to invoke an external method that caused an external event to trigger another ECA rule. ECA rules can also be used in rule engines that use variants of the Rete algorithm for rule processing. == ECA rule engines == Rulecore Concurrent Rules Apart Database Detect Invocation Rules ConceptBase ECArules

Confused deputy problem

In information security, a confused deputy is a computer program that is tricked by another program (with fewer privileges or less rights) into misusing its authority on the system. It is a specific type of privilege escalation. The confused deputy problem is often cited as an example of why capability-based security is important. Capability systems protect against the confused deputy problem, whereas access-control list–based systems do not. Such systems can mitigate the confused deputy problem by eliminating ambient authority, allowing programs to act only on resources for which they hold explicit capabilities, whereas access-control list–based systems are more susceptible to it. However, this protection depends on correct implementation; in formally verified capability systems such as seL4, it can be shown that the kernel enforces capability constraints correctly, preventing such behavior at the system level. == Example == In the original example of a confused deputy, there was a compiler program provided on a commercial timesharing service. Users could run the compiler and optionally specify a filename where it would write debugging output, and the compiler would be able to write to that file if the user had permission to write there. The compiler also collected statistics about language feature usage. Those statistics were stored in a file called "(SYSX)STAT", in the directory "SYSX". To make this possible, the compiler program was given permission to write to files in SYSX. But there were other files in SYSX: in particular, the system's billing information was stored in a file "(SYSX)BILL". A user ran the compiler and named "(SYSX)BILL" as the desired debugging output file. This produced a confused deputy problem. The compiler made a request to the operating system to open (SYSX)BILL. Even though the user did not have access to that file, the compiler did, so the open succeeded. The compiler wrote the compilation output to the file (here "(SYSX)BILL") as normal, overwriting it, and the billing information was destroyed. === The confused deputy === In this example, the compiler program is the deputy because it is acting at the request of the user. The program is seen as 'confused' because it was tricked into overwriting the system's billing file. Whenever a program tries to access a file, the operating system needs to know two things: which file the program is asking for, and whether the program has permission to access the file. In the example, the file is designated by its name, “(SYSX)BILL”. The program receives the file name from the user, but does not know whether the user had permission to write the file. When the program opens the file, the system uses the program's permission, not the user's. When the file name was passed from the user to the program, the permission did not go along with it; the permission was increased by the system silently and automatically. It is not essential to the attack that the billing file be designated by a name represented as a string. The essential points are that: the designator for the file does not carry the full authority needed to access the file; the program's own permission to access the file is used implicitly. == Other examples == A cross-site request forgery (CSRF) is an example of a confused deputy attack that uses the web browser to perform sensitive actions against a web application. A common form of this attack occurs when a web application uses a cookie to authenticate all requests transmitted by a browser. Using JavaScript, an attacker can force a browser into transmitting authenticated HTTP requests. The Samy computer worm used cross-site scripting (XSS) to turn the browser's authenticated MySpace session into a confused deputy. Using XSS the worm forced the browser into posting an executable copy of the worm as a MySpace message which was then viewed and executed by friends of the infected user. Clickjacking is an attack where the user acts as the confused deputy. In this attack a user thinks they are harmlessly browsing a website (an attacker-controlled website) but they are in fact tricked into performing sensitive actions on another website. An FTP bounce attack can allow an attacker to connect indirectly to TCP ports to which the attacker's machine has no access, using a remote FTP server as the confused deputy. Another example relates to personal firewall software. It can restrict Internet access for specific applications. Some applications circumvent this by starting a browser with instructions to access a specific URL. The browser has authority to open a network connection, even though the application does not. Firewall software can attempt to address this by prompting the user in cases where one program starts another which then accesses the network. However, the user frequently does not have sufficient information to determine whether such an access is legitimate—false positives are common, and there is a substantial risk that even sophisticated users will become habituated to clicking "OK" to these prompts. Not every program that misuses authority is a confused deputy. Sometimes misuse of authority is simply a result of a program error. The confused deputy problem occurs when the designation of an object is passed from one program to another, and the associated permission changes unintentionally, without any explicit action by either party. It is insidious because neither party did anything explicit to change the authority. Another example is when an administrator authorizes an AI agent to act on their behalf, and that AI subsequently delegates authority to another AI agent neither vetted nor authorized by the original administrator. The unvetted AI can then act without permissions or oversight from the original developer. == Solutions == In some systems it is possible to ask the operating system to open a file using the permissions of another client. This solution has some drawbacks: It requires explicit attention to security by the server. A naive or careless server might not take this extra step. It becomes more difficult to identify the correct permission if the server is in turn the client of another service and wants to pass along access to the file. It requires the client to trust the server to not abuse the borrowed permissions. Note that intersecting the server and client's permissions does not solve the problem either, because the server may then have to be given very wide permissions (all of the time, rather than those needed for a given request) in order to act for arbitrary clients. The simplest way to solve the confused deputy problem is to bundle together the designation of an object and the permission to access that object. This is exactly what a capability is. Using capability security in the compiler example, the client would pass to the server a capability to the output file, such as a file descriptor, rather than the name of the file. Since it lacks a capability to the billing file, it cannot designate that file for output. In the cross-site request forgery example, a URL supplied "cross"-site would include its own authority independent of that of the client of the web browser.

Language model benchmark

A language model benchmark is a standardized test designed to evaluate the performance of language models on various natural language processing tasks. These tests are intended for comparing different models' capabilities in areas such as language understanding, generation, and reasoning. Benchmarks generally consist of a dataset and corresponding evaluation metrics. The dataset provides text samples and annotations, while the metrics measure a model's performance on tasks like answering questions, text classification, and machine translation. These benchmarks are developed and maintained by academic institutions, research organizations, and industry players to track progress in the field. In addition to accuracy, the metrics can include throughput, energy efficiency, bias, trust, and sustainability. == Overview == === Types === Benchmarks may be described by the following adjectives, not mutually exclusive: Classical: These tasks are studied in natural language processing, even before the advent of deep learning. Examples include the Penn Treebank for testing syntactic and semantic parsing, as well as bilingual translation benchmarked by BLEU scores. Question answering: These tasks have a text question and a text answer, often multiple-choice. They can be open-book or closed-book. Open-book QA resembles reading comprehension questions, with relevant passages included as annotation in the question, in which the answer appears. Closed-book QA includes no relevant passages. Closed-book QA is also called open-domain question-answering. Before the era of large language models, open-book QA was more common, and understood as testing information retrieval methods. Closed-book QA became common since GPT-2 as a method to measure knowledge stored within model parameters. Omnibus: An omnibus benchmark combines many benchmarks, often previously published. It is intended as an all-in-one benchmarking solution. Reasoning: These tasks are usually in the question-answering format, but are intended to be more difficult than standard question answering. Multimodal: These tasks require processing not only text, but also other modalities, such as images and sound. Examples include OCR and transcription. Agency: These tasks are for a language-model–based software agent that operates a computer for a user, such as editing images, browsing the web, etc. Adversarial: A benchmark is "adversarial" if the items in the benchmark are picked specifically so that certain models do badly on them. Adversarial benchmarks are often constructed after state of the art (SOTA) models have saturated (achieved 100% performance) a benchmark, to renew the benchmark. A benchmark is "adversarial" only at a certain moment in time, since what is adversarial may cease to be adversarial as newer SOTA models appear. Public/Private: A benchmark might be partly or entirely private, meaning that some or all of the questions are not publicly available. The idea is that if a question is publicly available, then it might be used for training, which would be "training on the test set" and invalidate the result of the benchmark. Usually, only the guardians of the benchmark have access to the private subsets, and to score a model on such a benchmark, one must send the model weights, or provide API access, to the guardians. The boundary between a benchmark and a dataset is not sharp. Generally, a dataset contains three "splits": training, test, and validation. Both the test and validation splits are essentially benchmarks. In general, a benchmark is distinguished from a test/validation dataset in that a benchmark is typically intended to be used to measure the performance of many different models that are not trained specifically for doing well on the benchmark, while a test/validation set is intended to be used to measure the performance of models trained specifically on the corresponding training set. In other words, a benchmark may be thought of as a test/validation set without a corresponding training set. Conversely, certain benchmarks may be used as a training set, such as the English Gigaword or the One Billion Word Benchmark, which in modern language is just the negative log-likelihood loss on a pretraining set with 1 billion words. Indeed, the distinction between benchmark and dataset in language models became sharper after the rise of the pretraining paradigm, whereby a model is first trained on massive, unlabeled datasets to learn general language patterns, syntax, and knowledge (pretraining), and the base model is then adapted to specific, downstream tasks using smaller, labeled datasets (fine-tuning). === Lifecycle === Generally, the life cycle of a benchmark consists of the following steps: Inception: A benchmark is published. It can be simply given as a demonstration of the power of a new model (implicitly) that others then picked up as a benchmark, or as a benchmark that others are encouraged to use (explicitly). Growth: More papers and models use the benchmark, and the performance on the benchmark grows. Maturity, degeneration or deprecation: A benchmark may be saturated, after which researchers move on to other benchmarks. Progress on the benchmark may also be neglected as the field moves to focus on other benchmarks. Renewal: A saturated benchmark can be upgraded to make it no longer saturated, allowing further progress. === Construction === Like datasets, benchmarks are typically constructed by several methods, individually or in combination: Web scraping: Ready-made question-answer pairs may be scraped online, such as from websites that teach mathematics and programming. Conversion: Items may be constructed programmatically from scraped web content, such as by blanking out named entities from sentences, and asking the model to fill in the blank. This was used for making the CNN/Daily Mail Reading Comprehension Task. Crowd sourcing: Items may be constructed by paying people to write them, such as on Amazon Mechanical Turk. This was used for making the MCTest. === Evaluation === Generally, benchmarks are fully automated. This limits the questions that can be asked. For example, with mathematical questions, "proving a claim" would be difficult to automatically check, while "calculate an answer with a unique integer answer" would be automatically checkable. With programming tasks, the answer can generally be checked by running unit tests, with an upper limit on runtime. The benchmark scores are of the following kinds: For multiple choice or cloze questions, common scores are accuracy (frequency of correct answer), precision, recall, F1 score, etc. pass@n: The model is given n {\displaystyle n} attempts to solve each problem. If any attempt is correct, the model earns a point. The pass@n score is the model's average score over all problems. k@n: The model makes n {\displaystyle n} attempts to solve each problem, but only k {\displaystyle k} attempts out of them are selected for submission. If any submission is correct, the model earns a point. The k@n score is the model's average score over all problems. cons@n: The model is given n {\displaystyle n} attempts to solve each problem. If the most common answer is correct, the model earns a point. The cons@n score is the model's average score over all problems. Here "cons" stands for "consensus" or "majority voting". The pass@n score can be estimated more accurately by making N > n {\displaystyle N>n} attempts, and use the unbiased estimator 1 − ( N − c n ) ( N n ) {\displaystyle 1-{\frac {\binom {N-c}{n}}{\binom {N}{n}}}} , where c {\displaystyle c} is the number of correct attempts. For less well-formed tasks, where the output can be any sentence, there are the following commonly used scores including BLEU ROUGE, METEOR, NIST, word error rate, LEPOR, CIDEr, and SPICE. === Issues === error: Some benchmark answers may be wrong. ambiguity: Some benchmark questions may be ambiguously worded. subjective: Some benchmark questions may not have an objective answer at all. This problem generally prevents creative writing benchmarks. Similarly, this prevents benchmarking writing proofs in natural language, though benchmarking proofs in a formal language is possible. open-ended: Some benchmark questions may not have a single answer of a fixed size. This problem generally prevents programming benchmarks from using more natural tasks such as "write a program for X", and instead uses tasks such as "write a function that implements specification X". inter-annotator agreement: Some benchmark questions may be not fully objective, such that even people would not agree with 100% on what the answer should be. This is common in natural language processing tasks, such as syntactic annotation. shortcut: Some benchmark questions may be easily solved by an "unintended" shortcut. For example, in the SNLI benchmark, having a negative word like "not" in the second sentence is a strong signal for the "Contradiction" category, regardless of what the se

T-vertices

T-vertices is a term used in computer graphics to describe a problem that can occur during mesh refinement or mesh simplification. The most common case occurs in naive implementations of continuous level of detail, where a finer-level mesh is "sewn" together with a coarser-level mesh by simply aligning the finer vertices on the edges of the coarse polygons. The result is a continuous mesh, however due to the nature of the z-buffer and certain lighting algorithms such as Gouraud shading, visual artifacts can often be detected. Some modeling algorithms such as subdivision surfaces will fail when a model contains T-vertices.