AI Grammar Paraphrase Generator

AI Grammar Paraphrase Generator — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Evaluation of binary classifiers

  Evaluation of a binary classifier typically assigns a numerical value, or values, to a classifier that represent its accuracy. An example is error rate, which measures how frequently the classifier makes a mistake. There are many metrics that can be used; different fields have different preferences. For example, in medicine sensitivity and specificity are often used, while in computer science precision and recall are preferred. An important distinction is between metrics that are independent of the prevalence or skew (how often each class occurs in the population), and metrics that depend on the prevalence – both types are useful, but they have very different properties. Often, evaluation is used to compare two methods of classification, so that one can be adopted and the other discarded. Such comparisons are more directly achieved by a form of evaluation that results in a single unitary metric rather than a pair of metrics. == Contingency table == Given a data set, a classification (the output of a classifier on that set) gives two numbers: the number of positives and the number of negatives, which add up to the total size of the set. To evaluate a classifier, one compares its output to another reference classification – ideally a perfect classification, but in practice the output of another gold standard test – and cross tabulates the data into a 2×2 contingency table, comparing the two classifications. One then evaluates the classifier relative to the gold standard by computing summary statistics of these 4 numbers. Generally these statistics will be scale invariant (scaling all the numbers by the same factor does not change the output), to make them independent of population size, which is achieved by using ratios of homogeneous functions, most simply homogeneous linear or homogeneous quadratic functions. Say we test some people for the presence of a disease. Some of these people have the disease, and our test correctly says they are positive. They are called true positives (TP). Some have the disease, but the test incorrectly claims they don't. They are called false negatives (FN). Some don't have the disease, and the test says they don't – true negatives (TN). Finally, there might be healthy people who have a positive test result – false positives (FP). These can be arranged into a 2×2 contingency table (confusion matrix), conventionally with the test result on the vertical axis and the actual condition on the horizontal axis. These numbers can then be totaled, yielding both a grand total and marginal totals. Totaling the entire table, the number of true positives, false negatives, true negatives, and false positives add up to 100% of the set. Totaling the columns (adding vertically) the number of true positives and false positives add up to 100% of the test positives, and likewise for negatives. Totaling the rows (adding horizontally), the number of true positives and false negatives add up to 100% of the condition positives (conversely for negatives). The basic marginal ratio statistics are obtained by dividing the 2×2=4 values in the table by the marginal totals (either rows or columns), yielding 2 auxiliary 2×2 tables, for a total of 8 ratios. These ratios come in 4 complementary pairs, each pair summing to 1, and so each of these derived 2×2 tables can be summarized as a pair of 2 numbers, together with their complements. Further statistics can be obtained by taking ratios of these ratios, ratios of ratios, or more complicated functions. The contingency table and the most common derived ratios are summarized below; see sequel for details. Note that the rows correspond to the condition actually being positive or negative (or classified as such by the gold standard), as indicated by the color-coding, and the associated statistics are prevalence-independent, while the columns correspond to the test being positive or negative, and the associated statistics are prevalence-dependent. There are analogous likelihood ratios for prediction values, but these are less commonly used, and not depicted above. == Pairs of metrics == Often accuracy is evaluated with a pair of metrics composed in a standard pattern. === Sensitivity and specificity === The fundamental prevalence-independent statistics are sensitivity and specificity. Sensitivity or True Positive Rate (TPR), also known as recall, is the proportion of people that tested positive and are positive (True Positive, TP) of all the people that actually are positive (Condition Positive, CP = TP + FN). It can be seen as the probability that the test is positive given that the patient is sick. With higher sensitivity, fewer actual cases of disease go undetected (or, in the case of the factory quality control, fewer faulty products go to the market). Specificity (SPC) or True Negative Rate (TNR) is the proportion of people that tested negative and are negative (True Negative, TN) of all the people that actually are negative (Condition Negative, CN = TN + FP). As with sensitivity, it can be looked at as the probability that the test result is negative given that the patient is not sick. With higher specificity, fewer healthy people are labeled as sick (or, in the factory case, fewer good products are discarded). The relationship between sensitivity and specificity, as well as the performance of the classifier, can be visualized and studied using the Receiver Operating Characteristic (ROC) curve. In theory, sensitivity and specificity are independent in the sense that it is possible to achieve 100% in both (such as in the red/blue ball example given above). In more practical, less contrived instances, however, there is usually a trade-off, such that they are inversely proportional to one another to some extent. This is because we rarely measure the actual thing we would like to classify; rather, we generally measure an indicator of the thing we would like to classify, referred to as a surrogate marker. The reason why 100% is achievable in the ball example is because redness and blueness is determined by directly detecting redness and blueness. However, indicators are sometimes compromised, such as when non-indicators mimic indicators or when indicators are time-dependent, only becoming evident after a certain lag time. The following example of a pregnancy test will make use of such an indicator. Modern pregnancy tests do not use the pregnancy itself to determine pregnancy status; rather, human chorionic gonadotropin is used, or hCG, present in the urine of gravid females, as a surrogate marker to indicate that a woman is pregnant. Because hCG can also be produced by a tumor, the specificity of modern pregnancy tests cannot be 100% (because false positives are possible). Also, because hCG is present in the urine in such small concentrations after fertilization and early embryogenesis, the sensitivity of modern pregnancy tests cannot be 100% (because false negatives are possible). === Positive and negative predictive values === In addition to sensitivity and specificity, the performance of a binary classification test can be measured with positive predictive value (PPV), also known as precision, and negative predictive value (NPV). The positive prediction value answers the question "If the test result is positive, how well does that predict an actual presence of disease?". It is calculated as TP/(TP + FP); that is, it is the proportion of true positives out of all positive results. The negative prediction value is the same, but for negatives, naturally. ==== Impact of prevalence on predictive values ==== Prevalence has a significant impact on prediction values. As an example, suppose there is a test for a disease with 99% sensitivity and 99% specificity. If 2000 people are tested and the prevalence (in the sample) is 50%, 1000 of them are sick and 1000 of them are healthy. Thus about 990 true positives and 990 true negatives are likely, with 10 false positives and 10 false negatives. The positive and negative prediction values would be 99%, so there can be high confidence in the result. However, if the prevalence is only 5%, so of the 2000 people only 100 are really sick, then the prediction values change significantly. The likely result is 99 true positives, 1 false negative, 1881 true negatives and 19 false positives. Of the 19+99 people tested positive, only 99 really have the disease – that means, intuitively, that given that a patient's test result is positive, there is only 84% chance that they really have the disease. On the other hand, given that the patient's test result is negative, there is only 1 chance in 1882, or 0.05% probability, that the patient has the disease despite the test result. === Precision and recall === Precision and recall can be interpreted as (estimated) conditional probabilities: Precision is given by P ( C = P | C ^ = P ) {\displaystyle P(C=P|{\hat {C}}=P)} while recall is given by P ( C ^ = P | C = P ) {\displaystyle P({\hat {C}}=P|C=P)} , where C ^ {\

  Read more →
• 

  CARE Principles for Indigenous Data Governance

  The CARE Principles for Indigenous Data Governance are a set of principles intended to guide open data projects in engaging Indigenous Peoples rights and interests. CARE was created in 2019 by the International Indigenous Data Sovereignty Interest Group, a group that is a part of the Research Data Alliance. It outlines collective rights related to open data in the context of the United Nations Declaration on the Rights of Indigenous Peoples and Indigenous data sovereignty. CARE is an acronym which stands for Collective Benefit, Authority to Control, Responsibility, Ethics. The CARE Principles are 'people and purpose-oriented, reflecting the crucial role of data in advancing Indigenous innovation and self-determination', and intended as a complement to the data-oriented perspective of other standards such as FAIR data (findable, accessible, interoperable, reusable). The CARE principles have been embedded into the Beta version of Standardised Data on Initiatives (STARDIT). CARE principles were the basis of a submission to the UN's Global Digital Compact.

  Read more →
• 

  Why We Post

  Why We Post is a research project funded by the European Research Council and launched in 2012 by Daniel Miller with the objective of examining the global impact of new social media. The study is based on ethnographic data collected through the course of 15 months in China, India, Turkey, Italy, United Kingdom, Trinidad, Chile and Brazil. The results of this project were released on 29 February 2016. This included the first three of eleven Open Access books (available via UCL Press), a five-week e-course (MOOC) on FutureLearn in English, also available in Chinese, Portuguese, Hindi, Tamil, Italian, Turkish, and Spanish on UCLeXtend. In addition a website containing key discoveries, stories and over 100 films is available in the same 8 languages.

  Read more →
• 

  Instapoetry

  Instapoetry is a style of poetry that emerged after the advent of social media, especially on Instagram. The term has been used to describe poems written specifically for being shared online, most commonly on Instagram, but also other platforms including Twitter, Tumblr, and TikTok. The style usually consists of short, direct lines in aesthetically pleasing fonts that are sometimes accompanied by an image or drawing, often without rhyme schemes or meter, and dealing with commonplace themes. Literary critics, poets, and writers have contended with Instapoetry's focus on brevity and plainness compared to traditional poetry, criticizing it for reproducing rather than subverting normative ideas on social media platforms that favor popularity and accessibility over craft and depth. == History == Instapoetry developed as a result of young, predominantly women, amateur poets sharing their output to expand their readership, who began using social media as their preferred method of distribution rather than traditional publishing methods. The term "Instapoetry" is a portmanteau of the words "Instagram" and "poetry," and was created by other writers trying to define and understand the new extension of "instant poetry" shared via social media, most prominently Instagram. In its most basic form, Instapoetry usually consists of bite-sized verses that consider political and social subjects such as immigration, domestic violence, sexual assault, love, culture, feminism, gun violence, war, racism, LGBTQ rights, and other social justice topics. All of these elements are usually made to fit social media feeds that are easily accessible through applications on smartphones. == Scholarship == Despite the diversity of poetry on Instagram, the Brazilian linguist Bruna Osaki Fazano found that shared "aspects of the compositional form, theme and style" mean that it can be understood as a specific genre. Camilla Holm Soelseth argues that taking on the platform-specific tasks of a social media creator is a prerequisite for being an Instapoet. Writing in Poetics Today, JuEunhae Knox combined quantitative and qualitative analysis to show that Instapoetry is a cohesive genre, in part because "the sheer volume and rapidity of content production in turn encourages posts that are not only visually appealing but also immediately recognizable as Instapoems". Instapoetry has been seen as a practice that serves as a form of self-staging for poets and "[crafts] authenticity". Eirik Vassenden describes the work of Norwegian poet Trygve Skaug as appearing to offer a "simple, almost direct access to the inner self". Vassenden writes that poems such as Rupi Kaur's "if you are not enough for yourself / you will never be enough / for someone else" are "authentic" to such an extent that they are not literary. Kiera Obbard describes how Rupi Kaur uses humour as a rhetorical device in her poetry performances to tell personal stories of trauma and challenge social inequalities. Scholars have also studied the work of specific Instapoets, such as Rupi Kaur, R.M. Drake, Aja Monet, Yrsa Daley-Ward, Nayyirah Waheed, Atticus, Nikita Gill and Trygve Skaug. == Overview == Academics have shown appreciation for the way in which Instapoetry has stimulated interest in poetry in general. Meanwhile, it has been argued that since Instapoets avoid critical evaluations, academics, and the publishing industry, Instapoets qualify more as online celebrities than literary figures. Additionally, although Instapoetry has been characterized as anti-establishment, Alyson Miller noted traditional or even conservative views in the online posts of Instapoets in contrast with the activist views the style is associated with, and that there is a contradiction between "the extra-textual commentary surrounding Instapoetry, particularly by way of interviews and artistic statements, and the content of works which repeatedly reinscribe conservative, patriarchal, and heteronormative worldviews". Thom Young, a poet and high school English teacher, created a parody Instagram page as a way to mock Instapoets and their work, describing it as "fidget-spinner poetry. Like they're just scrolling on their devices, to read something instantly, while the libraries are empty. I think people today don't want to read anything that causes a whole lot of critical thinking." According to Johnathan Ford's piece in the Financial Times, as Instagram's algorithms have limited prospective Instapoets' reach-per-post, it has pushed them to pay to promote their material. Popular Instagram accounts will be promoted to the front of users' feeds, with the app's algorithm, in the view of critics, favoring the spread of bland, inauthentic, or clichéd content while preventing disciplined poetry from reaching new audiences. == Writers described as Instapoets == Rupi Kaur Atticus Amanda Lovelace Tyler Knott Gregson Najwa Zebian Lang Leav Nikita Gill Upile Chisala Tendai M. Shaba Donna Ashworth Trista Mateer

  Read more →
• 

  Condensation algorithm

  The condensation algorithm (Conditional Density Propagation) is a computer vision algorithm. The principal application is to detect and track the contour of objects moving in a cluttered environment. Object tracking is one of the more basic and difficult aspects of computer vision and is generally a prerequisite to object recognition. Being able to identify which pixels in an image make up the contour of an object is a non-trivial problem. Condensation is a probabilistic algorithm that attempts to solve this problem. The algorithm itself is described in detail by Isard and Blake in a publication in the International Journal of Computer Vision in 1998. One of the most interesting facets of the algorithm is that it does not compute on every pixel of the image. Rather, pixels to process are chosen at random, and only a subset of the pixels end up being processed. Multiple hypotheses about what is moving are supported naturally by the probabilistic nature of the approach. The evaluation functions come largely from previous work in the area and include many standard statistical approaches. The original part of this work is the application of particle filter estimation techniques. The algorithm's creation was inspired by the inability of Kalman filtering to perform object tracking well in the presence of significant background clutter. The presence of clutter tends to produce probability distributions for the object state which are multi-modal and therefore poorly modeled by the Kalman filter. The condensation algorithm in its most general form requires no assumptions about the probability distributions of the object or measurements. == Algorithm overview == The condensation algorithm seeks to solve the problem of estimating the conformation of an object described by a vector x t {\displaystyle \mathbf {x_{t}} } at time t {\displaystyle t} , given observations z 1 , . . . , z t {\displaystyle \mathbf {z_{1},...,z_{t}} } of the detected features in the images up to and including the current time. The algorithm outputs an estimate to the state conditional probability density p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} by applying a nonlinear filter based on factored sampling and can be thought of as a development of a Monte-Carlo method. p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} is a representation of the probability of possible conformations for the objects based on previous conformations and measurements. The condensation algorithm is a generative model since it models the joint distribution of the object and the observer. The conditional density of the object at the current time p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} is estimated as a weighted, time-indexed sample set { s t ( n ) , n = 1 , . . . , N } {\displaystyle \{s_{t}^{(n)},n=1,...,N\}} with weights π t ( n ) {\displaystyle \pi _{t}^{(n)}} . N is a parameter determining the number of sample sets chosen. A realization of p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} is obtained by sampling with replacement from the set s t {\displaystyle s_{t}} with probability equal to the corresponding element of π t {\displaystyle \pi _{t}} . The assumptions that object dynamics form a temporal Markov chain and that observations are independent of each other and the dynamics facilitate the implementation of the condensation algorithm. The first assumption allows the dynamics of the object to be entirely determined by the conditional density p ( x t | x t − 1 ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {x_{t-1}} )} . The model of the system dynamics determined by p ( x t | x t − 1 ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {x_{t-1}} )} must also be selected for the algorithm, and generally includes both deterministic and stochastic dynamics. The algorithm can be summarized by initialization at time t = 0 {\displaystyle t=0} and three steps at each time t: === Initialization === Form the initial sample set and weights by sampling according to the prior distribution. For example, specify as Gaussian and set the weights equal to each other. === Iterative procedure === Sample with replacement N {\displaystyle N} times from the set { s 0 ( n ) , n = 1 , . . . , N } {\displaystyle \{s_{0}^{(n)},n=1,...,N\}} with probability { π 0 ( n ) , n = 1 , . . . , N } {\displaystyle \{\pi _{0}^{(n)},n=1,...,N\}} to generate a realization of p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} . Apply the learned dynamics p ( x t | x t − 1 ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {x_{t-1}} )} to each element of this new set, to generate a new set { s t ( n ) } {\displaystyle \{s_{t}^{(n)}\}} . To take into account the current observation z t {\displaystyle \mathbf {z_{t}} } , set π t ( n ) = p ( z t | s ( n ) ) ∑ j = 1 N p ( z t | s ( j ) ) {\displaystyle \pi _{t}^{(n)}={\frac {p(\mathbf {z_{t}} |s^{(n)})}{\sum _{j=1}^{N}p(\mathbf {z_{t}} |s^{(j)})}}} for each element { s t ( n ) } {\displaystyle \{s_{t}^{(n)}\}} . This algorithm outputs the probability distribution p ( x t | z 1 , . . . , z t ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {z_{1},...,z_{t}} )} which can be directly used to calculate the mean position of the tracked object, as well as the other moments of the tracked object. Cumulative weights can instead be used to achieve a more efficient sampling. == Implementation considerations == Since object-tracking can be a real-time objective, consideration of algorithm efficiency becomes important. The condensation algorithm is relatively simple when compared to the computational intensity of the Ricatti equation required for Kalman filtering. The parameter N {\displaystyle N} , which determines the number of samples in the sample set, will clearly hold a trade-off in efficiency versus performance. One way to increase efficiency of the algorithm is by selecting a low degree of freedom model for representing the shape of the object. The model used by Isard 1998 is a linear parameterization of B-splines in which the splines are limited to certain configurations. Suitable configurations were found by analytically determining combinations of contours from multiple views, of the object in different poses, and through principal component analysis (PCA) on the deforming object. Isard and Blake model the object dynamics p ( x t | x t − 1 ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {x_{t-1}} )} as a second order difference equation with deterministic and stochastic components: p ( x t | x t − 1 ) ∝ e − 1 2 | | B − 1 ( ( x t − x ¯ ) − A ( x t − 1 − x ¯ ) ) | | 2 ) {\displaystyle p(\mathbf {x_{t}} |\mathbf {x_{t-1}} )\propto e^{-{\frac {1}{2}}||B^{-1}((\mathbf {x_{t}} -\mathbf {\bar {x}} )-A(\mathbf {x_{t-1}} -\mathbf {\bar {x}} ))||^{2})}} where x ¯ {\displaystyle \mathbf {\bar {x}} } is the mean value of the state, and A {\displaystyle A} , B {\displaystyle B} are matrices representing the deterministic and stochastic components of the dynamical model respectively. A {\displaystyle A} , B {\displaystyle B} , and x ¯ {\displaystyle \mathbf {\bar {x}} } are estimated via Maximum Likelihood Estimation while the object performs typical movements. The observation model p ( z | x ) {\displaystyle p(\mathbf {z} |\mathbf {x} )} cannot be directly estimated from the data, requiring assumptions to be made in order to estimate it. Isard 1998 assumes that the clutter which may make the object not visible is a Poisson random process with spatial density λ {\displaystyle \lambda } and that any true target measurement is unbiased and normally distributed with standard deviation σ {\displaystyle \sigma } . The basic condensation algorithm is used to track a single object in time. It is possible to extend the condensation algorithm using a single probability distribution to describe the likely states of multiple objects to track multiple objects in a scene at the same time. Since clutter can cause the object probability distribution to split into multiple peaks, each peak represents a hypothesis about the object configuration. Smoothing is a statistical technique of conditioning the distribution based on both past and future measurements once the tracking is complete in order to reduce the effects of multiple peaks. Smoothing cannot be directly done in real-time since it requires information of future measurements. == Applications == The algorithm can be used for vision-based robot localization of mobile robots. Instead of tracking the position of an object in the scene, however, the position of the camera platform is tracked. This allows the camera platform to be globally localized given a visual map of the environment. Extensions of the condensation algorithm have also been used to recognize human gestures in image sequences. This application of the condensation algorithm impacts the ran

  Read more →
• 

  Shorty Awards

  The Shorty Awards (also known as "The Shortys") are awards for outstanding and innovative work in digital and social media content by brands, advertising agencies, and creators. The awards, which generally focus on short-term content, honor achievements in content creation on Twitter, Facebook, YouTube, Instagram, TikTok, Twitch, and other social networking sites. The Shorty Awards began in 2008 and initially recognized achievements by independent creators on Twitter, with the first formal awards ceremony occurring in February 2009. Since then, the awards, which are now awarded each spring, have shifted their focus to recognize content across numerous platforms. Entrant work is judged on the merits of excellence in creativity, strategy, and engagement by the Real Time Academy, a group of industry professionals selected by the Shorty Awards on the basis of their professional reputations, industry knowledge, and personal achievements (which may include previous Shorty wins). An additional public voting component, known as Audience Honor Voting, is also used to select Shorty Awards contenders. Notable Shorty Award winners include Malala Yousafzai, Trevor Noah, Michelle Obama, Conan O’Brien, Lady Gaga, Bill Nye, Jacob Reed, and Lizzo. Brands and organizations such as Chipotle, Duolingo, Marvel Studios, HBO, Red Bull, Airbnb, Nestle, BMW, UNICEF and the Human Rights Campaign have also been awarded. The Shorty Awards also produces an annual award program called The Shorty Impact Awards, a competition dedicated to showcasing digital and social media-based projects by brands, agencies, and organizations that seek to make the world a better place. == List of ceremonies == == 1st Shorty Awards == The awards were created in 2008 by tech entrepreneurs Greg Galant, Adam Varga, and Lee Semel of Sawhorse Media. They invited Twitter account holders to nominate the best Twitter users in general categories such as humor, news, food, and design. Winners were chosen by more than 30,000 Twitter users during the voting period. The founders of Twitter first heard about the awards after the contest had gotten underway and expressed support for it. The first Shorty Awards ceremony was held on February 11, 2009, at the Galapagos Art Space in Brooklyn, New York. Approximately 300 people attended the event. The event was hosted by CNN anchor Rick Sanchez and featured appearances by prominent Twitter users MC Hammer and Gary Vaynerchuk and a video appearance by Shaquille O'Neal. The awards, in 26 categories, were voted on by Twitter users. == 2nd Shorty Awards == Voting for the second Shorty Awards opened in January 2010 in 26 official categories. A Real-Time Photo of the Year category was added to the list of official categories for the first time, recognizing the best photo posted to services such as Twitpic, Yfrog, or Facebook. The second Shorty Awards competition introduced a panel of judges called the Real-Time Academy of Short Form Arts & Sciences whose members were Craig Newmark, David Pogue, Kurt Andersen, Caterina Fake, Joi Ito, Frank Moss, Alberto Ibargüen, Sreenath Sreenivasan, MC Hammer, Alyssa Milano and Jimmy Wales. After public nominations determined the finalists, the academy decided on the winners. Winners were announced at a ceremony held in the Times Center in The New York Times building in Manhattan that was also streamed online. The ceremony was hosted by CNN anchor Rick Sanchez, who presented awards in the official categories as well as the newly added Real-Time Photo of the Year and a special humanitarian award. == 3rd Shorty Awards == The nomination period for the third annual Shorty Awards opened in January 2011 and ran through February 11, 2011, except for new categories that had extended nomination deadlines. There were 30 official categories and five special categories. In addition to Real-Time Photo of the Year, for the first time the awards accepted nominations for Foursquare Mayor of the Year, Foursquare Location of the Year, Microblog of the Year on Tumblr, and a Connecting People award. The awards also introduced new Shorty Industry Awards to recognize the best uses of social media by brands and agencies. Winners were announced at a ceremony on March 28, 2011, hosted by Aasif Mandvi in the Times Center. Other Shorty Awards presenters were scheduled to include Kiefer Sutherland, Jerry Stiller, Anne Meara, Stephen Wallem, Miss USA Rima Fakih, and Miss Teen USA Kamie Crawford. == 4th Shorty Awards == The 4th Annual Shorty Awards featured Ricky Gervais and Tiffani Thiessen. 1.6 million tweeted nominations were made across all the categories to honor the top users on Twitter, Facebook, Tumblr, Foursquare, YouTube and other internet platforms. == 5th Shorty Awards == The 5th Annual Shorty Awards ceremony featured Felicia Day, James Urbaniak, Kristian Nairn, Hannibal Buress, Carrie Keagan, Chris Hardwick, David Karp and Coco Rocha. 2.4 million tweeted nominations were made across all the categories to honor the top users on Twitter, Facebook, Tumblr, Foursquare, YouTube and other internet sites. == 6th Shorty Awards == The ceremony took place on April 7, 2014, at the New York TimesCenter and was hosted by Comedian Natasha Leggero. The show included appearances by Patton Oswalt, Jamie Oliver, Kristen Bell, Jerry Seinfeld, Moshe Kasher, Julie Klausner, Erin Brady, Guy Kawasaki, Matt Walsh, Retta, Us the Duo, Big Boi, Gilbert Gottfried, Thomas Middleditch, Billie Jean King and Leandra Medine. Winners included Jerry Seinfeld and Will Ferrell. == 7th Shorty Awards == The Seventh Annual Shorty Awards was hosted by comedian Rachel Dratch and took place on April 20, 2015, at The Times Center in NYC. The Real-Time Academy, the judging body of the Shortys, tripled in size for the 7th annual Awards and included Alton Brown, Mamrie Hart, Nikki Glaser, OK Go, The Fine Bros, Debbie Sterling, Dan Savage, Deena Varshavskaya and Palmer Luckey. Panic! at the Disco was the musical guest at the ceremony. On-stage presenters included Kevin Jonas, Bill Nye, Bella Thorne, Wyclef Jean, Emily Kinney and Tyler Oakley. == 8th Shorty Awards == The Eighth Annual Shorty Awards were held in NYC at the TimesCenter on April 11, 2016. They were hosted by YouTuber, Writer and Comedian Mamrie Hart with musical performances from Nico & Vinz. Winners of the night included Bill Wurtz, DJ Khaled, Misty Copeland, Casey Neistat, Dwayne Johnson, Hannah Hart, Troye Sivan, Baddie Winkle, Kevin Hart, Taraji P. Henson, King Bach, and Zach King. == 9th Shorty Awards == The Ninth Annual Shorty Awards were held in NYC at the PlayStation Theater on April 23, 2017. They were hosted by two-time Emmy Award winner Tony Hale with a musical performance by Lizzo. Winners of the night included Bill Nye, Shay Mitchell, Doug the Pug, Gigi Gorgeous, Simone Biles, Mara Wilson, Gaten Matarazzo and Chrissy Teigen. == 10th Shorty Awards == The 10th Annual Shorty Awards, took place on April 15, 2018, at the PlayStation Theater, New York City. The ceremony was hosted by actress, singer, and songwriter Keke Palmer with a musical performance by Betty Who. == 11th Shorty Awards == The 11th Annual Shorty Awards were held on May 5, 2019, in New York City at the PlayStation Theater. The ceremony was hosted by American actress and comedian Kathy Griffin, with a musical performance by Tank and the Bangas. == 12th Shorty Awards == The 12th Annual Shorty Awards were held on May 3, 2020. Due to the COVID-19 pandemic, the ceremony took place online for the first time, with presenters and award winners filming from their own homes. The ceremony was hosted by actor J.B. Smoove and featured a remixed performance of Trap Queen by Fetty Wap. Award winners included Jack Stauber, Supercar Blondie, Rose and Rosie, and Greta Thunberg. == 13th Shorty Awards == The 13th Annual Shorty Awards took place from April 26 to May 14, 2021. The ceremony was hosted on different social media platforms, such as Instagram and Clubhouse, to create a more tailored experience. Winners were announced from May 11 to May 14, with 10 winners being revealed each hour from 1 to 4 p.m. EST on the Shorty Awards Instagram account. == 14th Shorty Awards == The 14th Annual Shorty Awards were held virtually on May 15, 2022, honoring the best in social media and digital content. Hosted by Jay Shetty, the event recognized influencers, brands, and organizations across various categories, celebrating excellence in digital storytelling and innovative online campaigns. Notable winners included Tabitha Brown for her food content and the D'Amelio Family for their contributions to family and parenting content. The event highlighted the role of digital media in connecting and inspiring audiences during challenging times. == 15th Shorty Awards == The 15th Annual Shorty Awards celebrated the best in social media and digital content on May 24, 2023, at Tribeca 360° in New York City. Hosted by Jay Pharoah, the event honored creators, brands, and organizations ac

  Read more →
• 

  Personal web page

  Personal web pages are World Wide Web pages created by an individual to contain content of a personal nature rather than content pertaining to a company, organization or institution. Personal web pages are primarily used for informative or entertainment purposes but can also be used for personal career marketing (by containing a list of the individual's skills, experience and a CV), social networking with other people with shared interests, or as a space for personal expression. These terms do not usually refer to just a single "page" or HTML file, but to a website—a collection of webpages and related files under a common URL or Web address. In strictly technical terms, a site's actual home page (index page) often only contains sparse content with some catchy introductory material and serves mostly as a pointer or table of contents to the more content-rich pages inside, such as résumés, family, hobbies, family genealogy, a web log/diary ("blog"), opinions, online journals and diaries or other writing, examples of written work, digital audio sound clips, digital video clips, digital photos, or information about a user's other interests. Many personal pages only include information of interest to friends and family of the author. However, some webpages set up by hobbyists or enthusiasts of certain subject areas can be valuable topical web directories. == History == In the 1990s, most Internet service providers (ISPs) provided a free small personal, user-created webpage along with free Usenet News service. These were all considered part of full Internet service. Also several free web hosting services such as GeoCities provided free web space for personal web pages. These free web hosting services would typically include web-based site management and a few pre-configured scripts to easily integrate an input form or guestbook script into the user's site. Early personal web pages were often called "home pages" and were intended to be set as a default page in a web browser's preferences, usually by their owner. These pages would often contain links, to-do lists, and other information their author found useful. In the days when search engines were in their infancy, these pages (and the links they contained) could be an important resource in navigating the web. Since the early 2000s, the rise of blogging and the development of user friendly web page designing software made it easier for amateur users who did not have computer programming or website designer training to create personal web pages. Some website design websites provided free ready-made blogging scripts, where all the user had to do was input their content into a template. At the same time, a personal web presence became easier with the increased popularity of social networking services, some with blogging platforms such as LiveJournal and Blogger. These websites provided an attractive and easy-to-use content management system for regular users. Most of the early personal websites were Web 1.0 style, in which a static display of text and images or photos was displayed to individuals who came to the page. About the only interaction that was possible on these early websites was signing the virtual "guestbook". With the collapse of the dot-com bubble in the late 1990s, the ISP industry consolidated, and the focus of web hosting services shifted away from the surviving ISP companies to independent Internet hosting services and to ones with other affiliations. For example, many university departments provided personal pages for professors and television broadcasters provided them for their on-air personalities. These free webpages served as a perquisite ("perk") for staff, while at the same time boosting the Web visibility of the parent organization. Web hosting companies either charge a monthly fee, or provide service that is "free" (advertising based) for personal web pages. These are priced or limited according to the total size of all files in bytes on the host's hard drive, or by bandwidth, (traffic), or by some combination of both. For those customers who continue to use their ISP for these services, national ISPs commonly continue to provide both disk space and help including ready-made drop-in scripts. With the rise of Web 2.0-style websites, both professional websites and user-created, amateur websites tended to contain interactive features, such as "clickable" links to online newspaper articles or favourite websites, the option to comment on content displayed on the website, the option to "tag" images, videos or links on the site, the option of "clicking" on an image to enlarge it or find out more information, the option of user participation for website guests to evaluate or review the pages, or even the option to create new user-generated content for others to see. A key difference between Web 1.0 personal webpages and Web 2.0 personal pages was while the former tended to be created by hackers, computer programmers and computer hobbyists, the latter were created by a much wider variety of users, including individuals whose main interests lay in hobbies or topics outside of computers (e.g., indie music fans, political activists, and social entrepreneurs). == Motivations == In a study done by Zinkhan, participants had four main reasons to create personal web pages. First, people use personal web pages as a portrayal of self, in a sense marketing themselves, since creators have the freedom to portray their own identities. Second, personal web pages are a way to interact with people who have similar interests as the creator, possible employers, or colleagues. Third, personal web pages can gain social acceptance with groups that the creator is interested in depending on the information that the creator reveals about themselves. Fourth, personal web pages can give creators a sense of connection to the world since these web pages are public and a way to introduce oneself to other people around the globe. People may maintain personal web pages to serve as a showcase for their skills in professional life, creative skills or self promotion of their business, charity or band. The use of personal web pages to display an individual's professional life has become more common in the 21st century. Mary Madden, an expert researcher on privacy and technology, did a study that found a tenth of American jobs require Personal web pages that advertise an individual online. Personal web pages have become a source of initial impression of possible employees used by employers. It can also be used to express opinions on issues ranging from news and politics to movies. Others may use their personal web page as a communication method. For example, an aspiring artist might give out business cards with their personal web page, and invite people to visit their page and see their artwork, "like" their page or sign their guestbook. A personal web page gives the owner generally more control on presence in search results and how they wish to be viewed online. It also allows more freedom in types and quantity of content than a social network profile offers, and can link various social media profiles with each other. It can be used to correct the record on something, or clear up potential confusion between you and someone with the same name. In the 2010s, some amateur writers, bands and filmmakers release digital versions of their stories, songs and short films online, with the aim of gaining an audience and becoming more well-known. While the huge number of aspiring artists posting their work online makes it unlikely for individuals and groups to become popular via the Internet, there are a small number of YouTube stars who were unknown until their online performances garnered them a huge audience. == Sites of academics == Academic professionals (especially at the college and university level), including professors and researchers, are often given online space for creating and storing personal web documents, including personal web pages, CVs and a list of their books, academic papers and conference presentations, on the websites of their employers. This goes back to the early decade of the World Wide Web and its original purpose of providing a quick and easy way for academics to share research papers and data. Researchers may have a personal website to share more information about themselves, about their academic activities and for sharing (unpublished) results of their research. This has been noted as part of the success of open-access repositories such as arXiv.

  Read more →
• 

  Airborne Networking

  An Airborne Network (AN) is the infrastructure owned by the United States Air Force that provides communication transport services through at least one node that is on a platform capable of flight. == Background == === Definition === The intent of the US Air Force's Airborne Network is to expand the Global Information Grid (GIG) to connect the three major domains of warfare: Air, Space, and Terrestrial. The Transformational Satellite Communications System network currently provides connectivity for all communication through space assets. The Combat Information Transport System and Theater Deployable Communications provide terrestrial connectivity for theatre based operations. The Airborne Network is engineered to utilize all airborne assets to connect with space and surface networks building a seamless communications platform across all domains. === Capabilities === The capabilities identified by this type of system are vastly beyond that of our current military. This system will enable the Air Force to provide a transportable network, flexible enough to communicate with any air, space, or ground asset in the area. The network will provide a beyond line-of-sight (LoS) communications infrastructure that can be packed up and moved in and out of the designated battlespace, enabling the military to have a reliable and secure communications network that extends globally. The network is designed to be flexible enough to provide the right communication and network packages for a specific region, mission, or technology. Operationally, The AN is designed to be self-forming, self-organizing, and self-generating, with nodes joining and leaving the network as they enter and exit a specific region. The network consists of dedicated tactical links, wideband air-to-air links, and ad hoc networks constructed by the Joint Tactical Radio System (JTRS) networking services. JTRS is a software-defined radio that will work with many existing military and civilian radios. It includes integrated encryption and Wideband Networking Software to create mobile ad hoc networks. It also provides system performance analysis and fault diagnostics automatically, reducing the demand for human intervention and network maintenance. === Intended Use === The AN was designed as the cornerstone for the new military doctrine known as Network Centric Warfare. This doctrine was developed to use information superiority to equip warfighters with more precise information enabling commanders and shooters to make smarter decisions faster. The AN contributes to Network Centric Warfare by enabling commanders to provide real-time information to warfighters in the air and on the ground. Warfighters can then utilize more information and make more educated decisions about how to act in a particular situation. Once the act has been carried out commanders will have immediate information about the result and can make judgments on how to continue. All-in-all the AN was designed to reduce the time necessary to identify a target, make clear and educated decisions to pull or not to pull the trigger, and assess battle == Topologies == There are four main network topologies that will be deployed and vary based on the placement of backbone and subnet class networks. === Space, Air, Ground Tether === Establishing a direct connection to another aircraft or ground node, via a point-to-point link for nodes within LOS or via a Satellite Communications (SATCOM) link for nodes that are beyond line-of-sight is known as tethering. SATCOM links provide connectivity to a network ground entry point. Strike aircraft that accompany C2 aircraft such as an AWACS are tethered via point-to-point links. Finally, C2 or intelligence, surveillance, and reconnaissnce (ISR) aircraft may connect via a LOS link directly to a network ground entry point. Each of these tethered alternatives works exactly like a hub or switch that has an entry point to a larger network and allows their connected users access to that network. === Flat Ad Hoc === A flat ad hoc topology refers to establishing nonpersistent network connections as needed among AN nodes that are present at a given time. With this network the nodes dynamically “discover” other nodes to which they can interconnect and form the network. The specific interconnections between the nodes are not planned in advance, but are made as opportunities arise. The nodes join and leave the network at will, continually changing connections to neighbor nodes based upon their location and mobility characteristics. === Tiered Ad Hoc === Ad hoc networks can be flat in the sense that all nodes are peers of each other in a single network, as discussed above, or they can dynamically organize themselves into hierarchical tiers such that higher tiers are used to move data between more localized subnets. This network topology can be compared to any conventional deployed network that utilizes routers, switches, and hubs to temporarily connect users. === Persistent Backbone === A network topology characterized by a persistent backbone is established using relatively persistent wideband connections among high-value platforms flying relatively stable orbits. It provides the connectivity between the tactical subnets which are considered edge networks relative to the backbone. This provides concentration points for connectivity to the space backbone as well as to terrestrial networks. This type of network topology is comparable to a conventional permanent network with established data trunks, routers, switches, and hubs to connect users. == Architecture == === Network Management === The platform management system enables operators to manage all on-board network elements. It interfaces and interoperates with the Airborne Network management system to enable operators to manage remote network elements in the airborne network. The network management system monitors the health of the network by passively testing the network for faults and latency. The system will also actively troubleshoot faults with probes to identify and isolate faulty connections, and enables operators to apply network parameters and security changes to all systems based on the status of the network. === Routing/Switching === Routing and switching enables data to be dynamically transmitted over the network to other nodes. Routing protocols must be able to identify nodes transmitted within their own platform and data to be sent to other platforms regardless of the current topology. The routing protocol must also provide seamless roaming by ensuring that no routed packets are lost when a node changes its point of attachment to the network. Maintaining scalability is important in routing as the network is constantly changing. The network must be able to function with numerous levels of platforms, varying numbers of fast moving platforms, and varying amounts of traffic per platform. Routers and switches will use metrics to determine the best paths to take when routing data. The routing protocol utilized for the AN will be an Adaptive Quality of Service routing protocol. === Gateways/Proxies === Gateways and proxies enable the connection numerous technology types regardless of age to communicate across the IP-based network. Gateways and proxies are essential in the operation of this network because so many different technologies are used to communicate in each domain. These systems will facilitate the transition of the legacy on-board infrastructure, transmission systems, tactical data link systems, and user applications to the objective airborne network systems. Therefore, they are only temporary until all platforms use a standardized IP radio for transmission. === Performance Enhancing Proxies === Performance Enhancing Proxies improve the performance of user applications running across the Airborne Network by countering wireless network impairments, such as limited bandwidth, long delays, high loss rates, and disruptions in network connections. Proxy systems are implemented between the user application and the network and can be used to improve performance at the application and transport functional layers of the OSI model. Some techniques that can be employed include: Compression: Data compression or header compression can be used to minimize the number of bits sent over the network. Data bundling: Smaller data packets can be combined (bundled) into a single large packet for transmission over the network. Caching: A local cache can be used to save and provide data objects that are requested multiple times, reducing transmissions over the network (and improving response times). Store and forward: Message queuing can be used to ensure message delivery to users who become disconnected from the network or are unable to connect to the network for a period of time. Once the platform connects, the stored messages are sent. Pipelining: Rather than opening several separate network connections pipelining can be used to share a single networ

  Read more →
• 

  Learnable function class

  In statistical learning theory, a learnable function class is a set of functions for which an algorithm can be devised to asymptotically minimize the expected risk, uniformly over all probability distributions. The concept of learnable classes are closely related to regularization in machine learning, and provides large sample justifications for certain learning algorithms. == Definition == === Background === Let Ω = X × Y = { ( x , y ) } {\displaystyle \Omega ={\mathcal {X}}\times {\mathcal {Y}}=\{(x,y)\}} be the sample space, where y {\displaystyle y} are the labels and x {\displaystyle x} are the covariates (predictors). F = { f : X ↦ Y } {\displaystyle {\mathcal {F}}=\{f:{\mathcal {X}}\mapsto {\mathcal {Y}}\}} is a collection of mappings (functions) under consideration to link x {\displaystyle x} to y {\displaystyle y} . L : Y × Y ↦ R {\displaystyle L:{\mathcal {Y}}\times {\mathcal {Y}}\mapsto \mathbb {R} } is a pre-given loss function (usually non-negative). Given a probability distribution P ( x , y ) {\displaystyle P(x,y)} on Ω {\displaystyle \Omega } , define the expected risk I P ( f ) {\displaystyle I_{P}(f)} to be: I P ( f ) = ∫ L ( f ( x ) , y ) d P ( x , y ) {\displaystyle I_{P}(f)=\int L(f(x),y)dP(x,y)} The general goal in statistical learning is to find the function in F {\displaystyle {\mathcal {F}}} that minimizes the expected risk. That is, to find solutions to the following problem: f ^ = arg ⁡ min f ∈ F I P ( f ) {\displaystyle {\hat {f}}=\arg \min _{f\in {\mathcal {F}}}I_{P}(f)} But in practice the distribution P {\displaystyle P} is unknown, and any learning task can only be based on finite samples. Thus we seek instead to find an algorithm that asymptotically minimizes the empirical risk, i.e., to find a sequence of functions { f ^ n } n = 1 ∞ {\displaystyle \{{\hat {f}}_{n}\}_{n=1}^{\infty }} that satisfies lim n → ∞ P ( I P ( f ^ n ) − inf f ∈ F I P ( f ) > ϵ ) = 0 {\displaystyle \lim _{n\rightarrow \infty }\mathbb {P} (I_{P}({\hat {f}}_{n})-\inf _{f\in {\mathcal {F}}}I_{P}(f)>\epsilon )=0} One usual algorithm to find such a sequence is through empirical risk minimization. === Learnable function class === We can make the condition given in the above equation stronger by requiring that the convergence is uniform for all probability distributions. That is: The intuition behind the more strict requirement is as such: the rate at which sequence { f ^ n } {\displaystyle \{{\hat {f}}_{n}\}} converges to the minimizer of the expected risk can be very different for different P ( x , y ) {\displaystyle P(x,y)} . Because in real world the true distribution P {\displaystyle P} is always unknown, we would want to select a sequence that performs well under all cases. However, by the no free lunch theorem, such a sequence that satisfies (1) does not exist if F {\displaystyle {\mathcal {F}}} is too complex. This means we need to be careful and not allow too "many" functions in F {\displaystyle {\mathcal {F}}} if we want (1) to be a meaningful requirement. Specifically, function classes that ensure the existence of a sequence { f ^ n } {\displaystyle \{{\hat {f}}_{n}\}} that satisfies (1) are known as learnable classes. It is worth noting that at least for supervised classification and regression problems, if a function class is learnable, then the empirical risk minimization automatically satisfies (1). Thus in these settings not only do we know that the problem posed by (1) is solvable, we also immediately have an algorithm that gives the solution. == Interpretations == If the true relationship between y {\displaystyle y} and x {\displaystyle x} is y ∼ f ∗ ( x ) {\displaystyle y\sim f^{}(x)} , then by selecting the appropriate loss function, f ∗ {\displaystyle f^{}} can always be expressed as the minimizer of the expected loss across all possible functions. That is, f ∗ = arg ⁡ min f ∈ F ∗ I P ( f ) {\displaystyle f^{}=\arg \min _{f\in {\mathcal {F}}^{}}I_{P}(f)} Here we let F ∗ {\displaystyle {\mathcal {F}}^{}} be the collection of all possible functions mapping X {\displaystyle {\mathcal {X}}} onto Y {\displaystyle {\mathcal {Y}}} . f ∗ {\displaystyle f^{}} can be interpreted as the actual data generating mechanism. However, the no free lunch theorem tells us that in practice, with finite samples we cannot hope to search for the expected risk minimizer over F ∗ {\displaystyle {\mathcal {F}}^{}} . Thus we often consider a subset of F ∗ {\displaystyle {\mathcal {F}}^{}} , F {\displaystyle {\mathcal {F}}} , to carry out searches on. By doing so, we risk that f ∗ {\displaystyle f^{}} might not be an element of F {\displaystyle {\mathcal {F}}} . This tradeoff can be mathematically expressed as In the above decomposition, part ( b ) {\displaystyle (b)} does not depend on the data and is non-stochastic. It describes how far away our assumptions ( F {\displaystyle {\mathcal {F}}} ) are from the truth ( F ∗ {\displaystyle {\mathcal {F}}^{}} ). ( b ) {\displaystyle (b)} will be strictly greater than 0 if we make assumptions that are too strong ( F {\displaystyle {\mathcal {F}}} too small). On the other hand, failing to put enough restrictions on F {\displaystyle {\mathcal {F}}} will cause it to be not learnable, and part ( a ) {\displaystyle (a)} will not stochastically converge to 0. This is the well-known overfitting problem in statistics and machine learning literature. == Example: Tikhonov regularization == A good example where learnable classes are used is the so-called Tikhonov regularization in reproducing kernel Hilbert space (RKHS). Specifically, let F ∗ {\displaystyle {\mathcal {F^{}}}} be an RKHS, and | | ⋅ | | 2 {\displaystyle ||\cdot ||_{2}} be the norm on F ∗ {\displaystyle {\mathcal {F^{}}}} given by its inner product. It is shown in that F = { f : | | f | | 2 ≤ γ } {\displaystyle {\mathcal {F}}=\{f:||f||_{2}\leq \gamma \}} is a learnable class for any finite, positive γ {\displaystyle \gamma } . The empirical minimization algorithm to the dual form of this problem is arg ⁡ min f ∈ F ∗ { ∑ i = 1 n L ( f ( x i ) , y i ) + λ | | f | | 2 } {\displaystyle \arg \min _{f\in {\mathcal {F}}^{}}\left\{\sum _{i=1}^{n}L(f(x_{i}),y_{i})+\lambda ||f||_{2}\right\}} This was first introduced by Tikhonov to solve ill-posed problems. Many statistical learning algorithms can be expressed in such a form (for example, the well-known ridge regression). The tradeoff between ( a ) {\displaystyle (a)} and ( b ) {\displaystyle (b)} in (2) is geometrically more intuitive with Tikhonov regularization in RKHS. We can consider a sequence of { F γ } {\displaystyle \{{\mathcal {F}}_{\gamma }\}} , which are essentially balls in F ∗ {\displaystyle {\mathcal {F^{}}}} with centers at 0. As γ {\displaystyle \gamma } gets larger, F γ {\displaystyle {\mathcal {F}}_{\gamma }} gets closer to the entire space, and ( b ) {\displaystyle (b)} is likely to become smaller. However we will also suffer smaller convergence rates in ( a ) {\displaystyle (a)} . The way to choose an optimal γ {\displaystyle \gamma } in finite sample settings is usually through cross-validation. == Relationship to empirical process theory == Part ( a ) {\displaystyle (a)} in (2) is closely linked to empirical process theory in statistics, where the empirical risk { ∑ i = 1 n L ( y i , f ( x i ) ) , f ∈ F } {\displaystyle \{\sum _{i=1}^{n}L(y_{i},f(x_{i})),f\in {\mathcal {F}}\}} are known as empirical processes. In this field, the function class F {\displaystyle {\mathcal {F}}} that satisfies the stochastic convergence are known as uniform Glivenko–Cantelli classes. It has been shown that under certain regularity conditions, learnable classes and uniformly Glivenko-Cantelli classes are equivalent. Interplay between ( a ) {\displaystyle (a)} and ( b ) {\displaystyle (b)} in statistics literature is often known as the bias-variance tradeoff. However, note that in the authors gave an example of stochastic convex optimization for General Setting of Learning where learnability is not equivalent with uniform convergence.

  Read more →
• 

  Business intelligence

  Business intelligence (BI) consists of strategies, methodologies, and technologies used by enterprises for data analysis and management of business information to inform business strategies and business operations. Common functions of BI technologies include reporting, online analytical processing, analytics, dashboard development, data mining, process mining, complex event processing, business performance management, benchmarking, text mining, predictive analytics, and prescriptive analytics. BI tools can handle large amounts of structured and sometimes unstructured data to help organizations identify, develop, and otherwise create new strategic business opportunities. They aim to allow for the easy interpretation of these big data. Identifying new opportunities and implementing an effective strategy based on insights is assumed to potentially provide businesses with a competitive market advantage and long-term stability, and help them take strategic decisions. Business intelligence can be used by enterprises to support a wide range of business decisions ranging from operational to strategic. Basic operating decisions include product positioning or pricing. Strategic business decisions involve priorities, goals, and directions at the broadest level. In all cases, business intelligence is considered most effective when it combines data from the market in which a company operates (external data) with data from internal company sources, such as financial and operational information. When integrated, external and internal data provide a comprehensive view that creates ‘intelligence’ not possible from any single data source alone. Among their many uses, business intelligence tools empower organizations to gain insight into new markets, to assess demand and suitability of products and services for different market segments, and to gauge the impact of marketing efforts. BI applications use data gathered from a data warehouse (DW) or from a data mart, and the concepts of BI and DW combine as "BI/DW" or as "BIDW". A data warehouse contains a copy of analytical data that facilitates decision support. == History == The earliest known use of the term business intelligence is in Richard Millar Devens' Cyclopædia of Commercial and Business Anecdotes (1865). Devens used the term to describe how the banker Sir Henry Furnese gained profit by receiving and acting upon information about his environment, prior to his competitors: Throughout Holland, Flanders, France, and Germany, he maintained a complete and perfect train of business intelligence. The news of the many battles fought was thus received first by him, and the fall of Namur added to his profits, owing to his early receipt of the news. The ability to collect and react accordingly based on the information retrieved, Devens says, is central to business intelligence. When Hans Peter Luhn, a researcher at IBM, used the term business intelligence in an article published in 1958, he employed the Webster's Dictionary definition of intelligence: "the ability to apprehend the interrelationships of presented facts in such a way as to guide action towards a desired goal." In 1989, Howard Dresner (later a Gartner analyst) proposed business intelligence as an umbrella term to describe "concepts and methods to improve business decision making by using fact-based support systems." It was not until the late 1990s that this usage was widespread. == Definition == According to Solomon Negash and Paul Gray, business intelligence (BI) can be defined as systems that combine: Data gathering Data storage Knowledge management with analysis to evaluate complex corporate and competitive information for presentation to planners and decision makers, with the objective of improving the timeliness and the quality of the input to the decision process." According to Forrester Research, business intelligence is "a set of methodologies, processes, architectures, and technologies that transform raw data into meaningful and useful information used to enable more effective strategic, tactical, and operational insights and decision-making." Under this definition, business intelligence encompasses information management (data integration, data quality, data warehousing, master-data management, text- and content-analytics, et al.). Therefore, Forrester refers to data preparation and data usage as two separate but closely linked segments of the business-intelligence architectural stack. Some elements of business intelligence are: Multidimensional aggregation and allocation Denormalization, tagging, and standardization Realtime reporting with analytical alert A method of interfacing with unstructured data sources Group consolidation, budgeting, and rolling forecasts Statistical inference and probabilistic simulation Key performance indicators optimization Version control and process management Open item management Forrester distinguishes this from the business-intelligence market, which is "just the top layers of the BI architectural stack, such as reporting, analytics, and dashboards." === Compared with competitive intelligence === Though the term business intelligence is sometimes a synonym for competitive intelligence (because they both support decision making), BI uses technologies, processes, and applications to analyze mostly internal, structured data and business processes while competitive intelligence gathers, analyzes, and disseminates information with a topical focus on company competitors. If understood broadly, competitive intelligence can be considered as a subset of business intelligence. === Compared with business analytics === Business intelligence and business analytics are sometimes used interchangeably, but there are alternate definitions. Thomas Davenport, professor of information technology and management at Babson College argues that business intelligence should be divided into querying, reporting, Online analytical processing (OLAP), an "alerts" tool, and business analytics. In this definition, business analytics is the subset of BI focusing on statistics, prediction, and optimization, rather than the reporting functionality. == Unstructured data == Business operations can generate a very large amount of data in the form of emails, memos, notes from call centers, news, user groups, chats, reports, web pages, presentations, image files, video files, and marketing material. According to Merrill Lynch, more than 85% of all business information exists in these forms; a company might only use such a document a single time. Because of the way it is produced and stored, this information is either unstructured or semi-structured. The management of semi-structured data is an unsolved problem in the information technology industry. According to projections from Gartner (2003), white-collar workers spend 30–40% of their time searching, finding, and assessing unstructured data. BI uses both structured and unstructured data. The former is easy to search, and the latter contains a large quantity of the information needed for analysis and decision-making. Because of the difficulty of properly searching, finding, and assessing unstructured or semi-structured data, organizations may not draw upon these vast reservoirs of information, which could influence a particular decision, task, or project. This can ultimately lead to poorly informed decision-making. Therefore, when designing a business intelligence/DW solution, the specific problems associated with semi-structured and unstructured data must be accommodated, as well as those associated with structured data. === Limitations of semi-structured and unstructured data === There are several challenges to developing BI with semi-structured data. According to Inmon & Nesavich, some of those are: Physically accessing unstructured textual data – unstructured data is stored in a huge variety of formats. Terminology – Among researchers and analysts, there is a need to develop standardized terminology. Volume of data – As stated earlier, up to 85% of all data exists as semi-structured data. Couple that with the need for word-to-word and semantic analysis. Searchability of unstructured textual data – A simple search on some data, e.g. apple, results in links where there is a reference to that precise search term. (Inmon & Nesavich, 2008) gives an example: "a search is made on the term felony. In a simple search, the term felony is used, and everywhere there is a reference to felony, a hit to an unstructured document is made. But a simple search is crude. It does not find references to crime, arson, murder, embezzlement, vehicular homicide, and such, even though these crimes are types of felonies". === Metadata === To solve problems with searchability and assessment of data, it is necessary to know something about the content. This can be done by adding context through the use of metadata. Many systems already capture some metadata (e.g. filename, author, size, etc.), but more usef

  Read more →
• 

  Data set (IBM mainframe)

  In the context of IBM mainframe computers in the IBM System/360 line and its successors, a data set (IBM preferred) or dataset is a computer file having a record organization. Use of this term began with, e.g., DOS/360 and OS/360, and is still used by their successors, including the current VSE and z/OS. Documentation for these systems historically preferred this term rather than file. A data set is typically stored on a direct access storage device (DASD) or magnetic tape, however unit record devices, such as punch card readers, card punches, line printers and page printers can provide input/output (I/O) for a data set (file). Data sets are not unstructured streams of bytes, but rather are organized in various logical record and block structures determined by the DSORG (data set organization), RECFM (record format), and other parameters. These parameters are specified at the time of the data set allocation (creation), for example with Job Control Language DD statements. Within a running program they are stored in the Data Control Block (DCB) or Access Control Block (ACB), which are data structures used to access data sets using access methods. Records in a data set may be fixed, variable, or “undefined” length. == Data set organization == For OS/360, the DCB's DSORG parameter specifies how the data set is organized. It may be CQ Queued Telecommunications Access Method (QTAM) in Message Control Program (MCP) CX Communications line group DA Basic Direct Access Method (BDAM) GS Graphics device for Graphics Access Method(GAM) IS Indexed Sequential Access Method (ISAM) MQ QTAM message queue in application PO Partitioned Organization PS Physical Sequential among others. Data sets on tape may only be DSORG=PS. The choice of organization depends on how the data is to be accessed, and in particular, how it is to be updated. Programmers utilize various access methods (such as QSAM or VSAM) in programs for reading and writing data sets. Access method depends on the given data set organization. == Record format (RECFM) == Regardless of organization, the physical structure of each record is essentially the same, and is uniform throughout the data set. This is specified in the DCB RECFM parameter. RECFM=F means that the records are of fixed length, specified via the LRECL parameter. RECFM=V specifies a variable-length record. V records when stored on media are prefixed by a Record Descriptor Word (RDW) containing the integer length of the record in bytes and flag bits. With RECFM=FB and RECFM=VB, multiple logical records are grouped together into a single physical block on tape or DASD. FB and VB are fixed-blocked, and variable-blocked, respectively. RECFM=U (undefined) is also variable length, but the length of the record is determined by the length of the block rather than by a control field. The BLKSIZE parameter specifies the maximum length of the block. RECFM=FBS could be also specified, meaning fixed-blocked standard, meaning all the blocks except the last one were required to be in full BLKSIZE length. RECFM=VBS, or variable-blocked spanned, means a logical record could be spanned across two or more blocks, with flags in the RDW indicating whether a record segment is continued into the next block and/or was continued from the previous one. This mechanism eliminates the need for using any "delimiter" byte value to separate records. Thus data can be of any type, including binary integers, floating-point, or characters, without introducing a false end-of-record condition. The data set is an abstraction of a collection of records, in contrast to files as unstructured streams of bytes. == Partitioned data set == A partitioned data set (PDS) is a data set containing multiple members, each of which holds a separate sub-data set, similar to a directory in other types of file systems. This type of data set is often used to hold load modules (old format bound executable programs), source program libraries (especially Assembler macro definitions), ISPF screen definitions, and Job Control Language. A PDS may be compared to a Zip file or COM Structured Storage. A Partitioned Data Set can only be allocated on a single volume and have a maximum size of 65,535 tracks. Besides members, a PDS contains also a directory. Each member can be accessed indirectly via the directory structure. Once a member is located, the data stored in that member are handled in the same manner as a PS (sequential) data set. Whenever a member is deleted, the space it occupied is unusable for storing other data. Likewise, if a member is re-written, it is stored in a new spot at the back of the PDS and leaves wasted “dead” space in the middle. The only way to recover “dead” space is to perform file compression. Compression, which is done using the IEBCOPY utility, moves all members to the front of the data space and leaves free usable space at the back. (Note that in modern parlance, this kind of operation might be called defragmentation or garbage collection; data compression nowadays refers to a different, more complicated concept.) PDS files can only reside on DASD, not on magnetic tape, in order to use the directory structure to access individual members. Partitioned data sets are most often used for storing multiple job control language files, utility control statements, and executable modules. An improvement of this scheme is a Partitioned Data Set Extended (PDSE or PDS/E, sometimes just libraries) introduced with DFSMSdfp for MVS/XA and MVS/ESA systems. A PDS/E library can store program objects or other types of members, but not both. BPAM cannot process a PDS/E containing program objects. PDS/E structure is similar to PDS and is used to store the same types of data. However, PDS/E files have a better directory structure which does not require pre-allocation of directory blocks when the PDS/E is defined (and therefore does not run out of directory blocks if not enough were specified). Also, PDS/E automatically stores members in such a way that compression operation is not needed to reclaim "dead" space. PDS/E files can only reside on DASD in order to use the directory structure to access individual members. == Generation Data Group == A Generation Data Group (GDG) is a group of non-VSAM data sets that are successive generations of historically-related data stored on an IBM mainframe (running OS/360 and its successors or DOS/360 and its successors). A GDG is usually cataloged. An individual member of the GDG collection is called a "Generation Data Set." The latter may be identified by an absolute number, ACCTG.OURGDG(1234), or a relative number: (-1) for the previous generation, (0) for the current one, and (+1) the next generation. A GDG specifies how many generations of a data set are to be kept and at what age a generation will be deleted. Whenever a new generation is created, the system checks whether one or more obsolete generations are to be deleted. The purpose of GDGs is to automate archival, using the command language JCL, the data set name given is generic. When DSN appears, the GDG data set appears along with the history number, where (0) is the most recent version (-1), (-2), ... are previous generations (+1) a new generation (see DD) Another use of GDGs is to be able to address all generations simultaneously within a JCL script without having to know the number of currently available generations. To do this, you have to omit the parentheses and the generation number in the JCL when specifying the dataset. === GDG JCL & features === Generation Data Groups are defined using either the BLDG statement of the IEHPROGM utility or the DEFINE GENERATIONGROUP statement of the newer IDCAMS utility, which allows setting various parameters. LIMIT(10) would limit the number of generations limit to 10. SCRATCH FOR (91) would retain each member, up to the limited#generations, at least 91 days. IDCAMS can also delete (and optionally uncatalog) a GDG. ==== Example ==== Creation of a standard GDG for five safety scopes, each at least 35 days old: Delete a standard GDG:

  Read more →
• 

  Stegomalware

  Stegomalware is a form of malicious software that leverages steganography techniques to conceal its code, configuration data, or command-and-control (C&C) communications within seemingly benign digital media such as images, audio files, videos, documents, or network traffic. It typically embeds encrypted or obfuscated payloads into digital media and only extracts and executes them at runtime, which makes traditional signature-based and sandbox-based detection significantly more difficult. Stegomalware has been observed in attacks ranging from advanced persistent threats (APTs) to financially motivated cybercrime, and is now the subject of dedicated academic surveys, research projects, and international law-enforcement initiatives. The key distinction between stegomalware and traditional obfuscated malware lies in the encoding location. After obfuscation, malicious code remains present within the executable and can theoretically be discovered through static analysis. In contrast, stegomalware hides the payload entirely within a cover medium (image, audio, etc.), remaining invisible until the malware dynamically extracts and executes it at runtime. == History == The term stegomalware was formally introduced by researchers Águila, Laskov, and others in the context of mobile malware and presented at the Inscrypt (Information Security and Cryptology) conference in 2014. This marked the first academic formalization of the concept, though earlier work had already identified that botnets and mobile malware could use steganography and covert channels for command-and-control communication over probabilistically unobservable channels. Since its introduction, stegomalware has evolved from a theoretical concern to a documented threat. In 2011, the APT operation known as "Operation Shady RAT" became one of the first documented cases of stegomalware in the wild, using digital images to hide Internet Protocol addresses and command-and-control server addresses. The same year, the Duqu malware (targeting industrial manufacturers) embedded victim data into JPEG image files before exfiltration, making the data transfer virtually undetectable to network-level security tools. From 2014 onwards, stegomalware became more prevalent in organized cybercrime and advanced persistent threat campaigns. Notable examples include Zeus/Zbot, which masked configuration data in images; Gatak/Stegoloader, which hid shellcode in PNG files; TeslaCrypt, which embedded C&C commands in JPEGs; and Cerber, which concealed ransomware payloads within images. By the 2010s, stegomalware had become established as a preferred evasion technique for espionage, financial theft, and ransomware distribution campaigns. Recent surveys (2020–2025) document that stegomalware has increasingly been exploited by adversaries targeting banks, enterprises, government agencies, educational institutions, and internet users via malvertising campaigns. The technique is now considered a sophisticated method of attack worthy of dedicated international law-enforcement attention. == Technical Characteristics and Definitions == Stegomalware operates through a three-component architecture: Stegotext (R): An innocent-looking digital asset (image, audio file, etc.) into which the malicious payload is embedded. Secret key (sk): A key used by the embedding and extraction algorithms, typically hardcoded into the malware. Payload (p): The actual malicious code, configuration data, or C&C commands hidden within the stegotext. The malware extracts the payload at runtime using the secret key and either executes it directly or uses it to download additional stages of the attack. Stegomalware can be classified into several types based on deployment method: Type 0 (Autonomous): Both the stegotext and extraction algorithm are embedded within the malware application itself. The malicious payload is extracted and executed locally without external communication. Type I (Update): The stegotext and secret key are downloaded from a remote server at runtime; only the extraction algorithm is included in the malware. This variant is more flexible, allowing attackers to push updated payloads. Type II (External Algorithm): Neither the stegotext nor the extraction algorithm are distributed with the malware; both are fetched from an attacker-controlled infrastructure, providing maximum flexibility and evasion. == Steganography techniques == === Spatial domain methods === Stegomalware predominantly uses steganographic methods designed for images, as images are the most common cover medium in the wild. The most basic spatial domain technique is Least Significant Bit (LSB) substitution, which replaces the least significant bits of pixel color values with payload bits. While simple and easy to implement, LSB is also relatively easy to detect through statistical analysis. More sophisticated spatial domain techniques include: HUGO (High Undetectable steGO) (2010): Minimizes detectable distortion by distributing the payload across multiple pixels, achieving embedding capacity with reduced statistical footprint. WOW (Wavelet Obtained Weights) (2012): Embeds data preferentially in textured regions of images where modifications are less perceptually noticeable. UNIWARD (Universal Wavelet Relative Distortion) (2014): Uses a universal distortion function applicable to multiple image formats, balancing payload capacity with undetectability. HILL (2014): Applies high-pass and low-pass filters to identify robust embedding regions. MiPOD (Minimizing the Power of Optimal Detector) (2016): Designed to minimize the power of theoretical optimal steganalysis detectors. === Transform domain methods === Transform domain techniques convert images into the frequency domain (e.g., using DCT or DWT) before embedding, allowing for more robust hiding in JPEG and other compressed formats: Embedding in DCT coefficients (used in JPEG compression) Embedding in DWT coefficients (used in lossless formats) Spread spectrum techniques, which distribute the payload across many frequency components Transform domain methods are generally more resistant to noise, compression, and image transformations than spatial methods. === Generative adversarial network (GAN) methods === Recent advances in machine learning have introduced GAN-based steganography, where a generative model produces stego images that minimize detectable artifacts: SGAN (Steganographic GAN) (2017): First GAN applied to steganography, using a generator, discriminator, and steganalysis network. ASDL-GAN (2017): Performs automatic steganographic distortion learning at the pixel level. SteganoGAN (2019): Improves upon earlier GAN models, achieving higher embedding capacity and robustness. HiGAN (Hiding Images GAN) (2020): Enables hiding one image within another while maintaining visual plausibility. GAN-based approaches are more resilient to standard steganalysis attacks but remain an emerging threat requiring further research. == Notable malware campaigns == Stegomalware has been documented in numerous high-profile cyber attacks and campaigns. Notable examples include: Operation Shady RAT (2011): Used digital images to hide command-and-control server addresses in targeted espionage. Duqu (2011): Embedded victim data into JPEG files to exfiltrate industrial control system information. Zeus/Zbot (2014): Masked banking configuration data inside JPEG files exploited via malvertising. Gatak/Stegoloader (2015): Hid shellcode in PNG files for software licensing attacks and bot command execution. TeslaCrypt (2015): Embedded C&C commands and ransomware keys in JPEG images. Cerber (2016): Concealed executable ransomware code in JPEG files distributed via phishing. DNSChanger (2016): Embedded malicious code in PNG files for DNS hijacking campaigns. Sundown Exploit Kit (2017): Distributed exploit code in PNG files via malvertising. AdGholas (2017): Used JPEG steganography to distribute ransomware via malvertising. Synccrypt (2017): Hidden ransomware components in JPEG-steganographic encrypted archives. ZeroT/PlugX (2017): Hid Remote Access Trojan payloads in BMP files for espionage. Loki Bot (2018): Concealed malware installers in JPEG and video files. Waterbug (APT28) (2019): Injected malicious DLLs into WAV audio files. Shlayer (macOS adware) (2019): Hid malicious URLs in JPEG files via malvertising. === Attack vectors === The most common attack vectors for stegomalware include: Phishing emails with malicious attachments or links Malvertising campaigns using malicious banner advertisements Exploit kits through compromised or malicious websites Legitimate application vulnerabilities (e.g., watering-hole attacks) Fake software distribution (cracked software, keygen tools) === Exploitation stages === Stegomalware typically serves one or more roles in attack lifecycles: Payload delivery: Stego images contain full executable code or shellcode. C&C communication: Hidden data contains server addresses or command instructio

  Read more →
• 

  Deductive language

  A deductive language is a computer programming language in which the program is a collection of predicates ('facts') and rules that connect them. Such a language is used to create knowledge based systems or expert systems which can deduce answers to problem sets by applying the rules to the facts they have been given. An example of a deductive language is Prolog, or its database-query cousin, Datalog. == History == As the name implies, deductive languages are rooted in the principles of deductive reasoning; making inferences based upon current knowledge. The first recommendation to use a clausal form of logic for representing computer programs was made by Cordell Green (1969) at Stanford Research Institute (now SRI International). This idea can also be linked back to the battle between procedural and declarative information representation in early artificial intelligence systems. Deductive languages and their use in logic programming can also be dated to the same year when Foster and Elcock introduced Absys, the first deductive/logical programming language. Shortly after, the first Prolog system was introduced in 1972 by Colmerauer through collaboration with Robert Kowalski. == Components == The components of a deductive language are a system of formal logic and a knowledge base upon which the logic is applied. === Formal Logic === Formal logic is the study of inference in regards to formal content. The distinguishing feature between formal and informal logic is that in the former case, the logical rule applied to the content is not specific to a situation. The laws hold regardless of a change in context. Although first-order logic is described in the example below to demonstrate the uses of a deductive language, no formal system is mandated and the use of a specific system is defined within the language rules or grammar. As input, a predicate takes any object(s) in the domain of interest and outputs either one of two Boolean values: true or false. For example, consider the sentences "Barack Obama is the 44th president" and "If it rains today, I will bring an umbrella". The first is a statement with an associated truth value. The second is a conditional statement relying on the value of some other statement. Either of these sentences can be broken down into predicates which can be compared and form the knowledge base of a deductive language. Moreover, variables such as 'Barack Obama' or 'president' can be quantified over. For example, take 'Barack Obama' as variable 'x'. In the sentence "There exists an 'x' such that if 'x' is the president, then 'x' is the commander in chief." This is an example of the existential quantifier in first order logic. Take 'president' to be the variable 'y'. In the sentence "For every 'y', 'y' is the leader of their nation." This is an example of the universal quantifier. === Knowledge Base === A collection of 'facts' or predicates and variables form the knowledge base of a deductive language. Depending on the language, the order of declaration of these predicates within the knowledge base may or may not influence the result of applying logical rules. Upon application of certain 'rules' or inferences, new predicates may be added to a knowledge base. As new facts are established or added, they form the basis for new inferences. As the core of early expert systems, artificial intelligence systems which can make decisions like an expert human, knowledge bases provided more information than databases. They contained structured data, with classes, subclasses, and instances. == Prolog == Prolog is an example of a deductive, declarative language that applies first- order logic to a knowledge base. To run a program in Prolog, a query is posed and based upon the inference engine and the specific facts in the knowledge base, a result is returned. The result can be anything appropriate from a new relation or predicate, to a literal such as a Boolean (true/false), depending on the engine and type system.

  Read more →
• 

  Instant messaging

  Instant messaging (IM) technology is a type of synchronous computer-mediated communication involving the immediate (real-time) transmission of messages between two or more parties over the Internet or another computer network. Originally involving simple text message exchanges, modern instant messaging applications and services (also variously known as instant messenger, messaging app, chat app, chat client, or simply a messenger) tend to also feature the exchange of multimedia, emojis, file transfer, VoIP (voice calling), and video chat capabilities. Instant messaging systems facilitate connections between specified known users (often using a contact list also known as a "buddy list" or "friend list") or in chat rooms, and can be standalone apps or integrated into a wider social media platform, or in a website where it can, for instance, be used for conversational commerce. Originally the term "instant messaging" was distinguished from "text messaging" by being run on a computer network instead of a cellular/mobile network, being able to write longer messages, real-time communication, presence ("status"), and being free (only cost of access instead of per SMS message sent). Instant messaging was pioneered in the early Internet era; the IRC protocol was the earliest to achieve wide adoption. Later in the 1990s, ICQ was among the first closed and commercialized instant messengers, and several rival services appeared afterwards as it became a popular use of the Internet. Beginning with its first introduction in 2005, BlackBerry Messenger became the first popular example of mobile-based IM, combining features of traditional IM and mobile SMS. Instant messaging remains very popular today; IM apps are the most widely used smartphone apps: in 2018 for instance there were 980 million monthly active users of WeChat and 1.3 billion monthly users of WhatsApp, the largest IM network. == Overview == Instant messaging (IM), sometimes also called "messaging" or "texting", consists of computer-based human communication between two users (private messaging) or more (chat room or "group") in real-time, allowing immediate receipt of acknowledgment or reply. This is in direct contrast to email, where conversations are not in real-time, and the perceived quasi-synchrony of the communications by the users (although many systems allow users to send offline messages that the other user receives when logging in). Earlier IM networks were limited to text-based communication, not dissimilar to mobile text messaging. As technology has moved forward, IM has expanded to include voice calling using a microphone, videotelephony using webcams, file transfer, location sharing, image and video transfer, voice notes, and other features. IM is conducted over the Internet or other types of networks (see also LAN messenger). Depending on the IM protocol, the technical architecture can be peer-to-peer (direct point-to-point transmission) or client–server (when all clients have to first connect to the central server). Primary IM services are controlled by their corresponding companies and usually follow the client-server model. At one point, the term "Instant Messenger" was a service mark of AOL Time Warner and could not be used in software not affiliated with AOL in the United States. For this reason, in April 2007, the instant messaging client formerly named Gaim (or gaim) announced that they would be renamed "Pidgin". === Clients === Modern IM services generally provide their own client, either a separately installed application or a browser-based client. They are normally centralised networks run by the servers of the platform's operators, unlike peer-to-peer protocols like XMPP. These usually only work within the same IM network, although some allow limited function with other services (see #Interoperability). Third-party client software applications exist that will connect with most of the major IM services. There is the class of instant messengers that uses the serverless model, which doesn't require servers, and the IM network consists only of clients. There are several serverless messengers: RetroShare, Tox, Bitmessage, Ricochet. See also: LAN messenger. Some examples of popular IM services today include Signal, Telegram, WhatsApp Messenger, WeChat, QQ Messenger, Viber, Line, and Snapchat. The popularity of certain apps greatly differ between different countries. Certain apps have an emphasis on certain uses - for example, Skype focuses on video calling, Slack focuses on messaging and file sharing for work teams, and Snapchat focuses on image messages. Some social networking services offer messaging services as a component of their overall platform, such as Facebook's Facebook Messenger, who also own WhatsApp. Others have a direct IM function as an additional adjunct component of their social networking platforms, like Instagram, Reddit, Tumblr, TikTok, Clubhouse and Twitter; this also includes for example dating websites, such as OkCupid or Plenty of Fish, and online gaming chat platforms. === Features === ==== Private and group messaging ==== Private chat allows users to converse privately with another person or a group. Privacy can also be enhanced in several ways, such as end-to-end encryption by default. Public and group chat features allow users to communicate with multiple people simultaneously. ==== Calling ==== Many major IM services and applications offer a call feature for user-to-user voice calls, conference calls, and voice messages. The call functionality is useful for professionals who utilize the application for work purposes and as a hands-free method. Videotelephony using a webcam is also possible by some. ==== Games and entertainment ==== Some IM applications include in-app games for entertainment. Yahoo! Messenger, for example, introduced these where users could play a game and viewed by friends in real-time. MSN Messenger featured a number of playable games within the interface. Facebook's Messenger has had a built-in option to play games with people in a chat, including games like Tetris and Blackjack. Discord features multiple games built inside the "activities" tab in voice channels. ==== Payments ==== A relatively new feature to instant messaging, peer-to-peer payments are available for financial tasks on top of communication. The lack of a service fee also makes these advantageous to financial applications. IM services such as Facebook Messenger and the WeChat 'super-app' for example offer a payment feature. == History == === Early systems === Though the term dates from the 1990s, instant messaging predates the Internet, first appearing on multi-user operating systems like Compatible Time-Sharing System (CTSS) and Multiplexed Information and Computing Service (Multics) in the mid-1960s. Initially, some of these systems were used as notification systems for services like printing, but quickly were used to facilitate communication with other users logged into the same machine. CTSS facilitated communication via text message for up to 30 people. Parallel to instant messaging were early online chat facilities, the earliest of which was Talkomatic (1973) on the PLATO system, which allowed 5 people to chat simultaneously on a 512 x 512 plasma display (5 lines of text + 1 status line per person). During the bulletin board system (BBS) phenomenon that peaked during the 1980s, some systems incorporated chat features which were similar to instant messaging; Freelancin' Roundtable was one prime example. The first such general-availability commercial online chat service (as opposed to PLATO, which was educational) was the CompuServe CB Simulator in 1980, created by CompuServe executive Alexander "Sandy" Trevor in Columbus, Ohio. As networks developed, the protocols spread with the networks. Some of these used a peer-to-peer protocol (e.g. talk, ntalk and ytalk), while others required peers to connect to a server (see talker and IRC). The Zephyr Notification Service (still in use at some institutions) was invented at MIT's Project Athena in the 1980s to allow service providers to locate and send messages to users. Early instant messaging programs were primarily real-time text, where characters appeared as they were typed. This includes the Unix "talk" command line program, which was popular in the 1980s and early 1990s. Some BBS chat programs (i.e. Celerity BBS) also used a similar interface. Modern implementations of real-time text also exist in instant messengers, such as AOL's Real-Time IM as an optional feature. In the latter half of the 1980s and into the early 1990s, the Quantum Link online service for Commodore 64 computers offered user-to-user messages between concurrently connected customers, which they called "On-Line Messages" (or OLM for short), and later "FlashMail." Quantum Link later became America Online and made AOL Instant Messenger (AIM, discussed later). While the Quantum Link client software ran on a Commodore 64, using only

  Read more →
• 

  IWARP

  iWARP is a computer networking protocol that implements remote direct memory access (RDMA) for efficient data transfer over Internet Protocol networks. Contrary to some accounts, iWARP is not an acronym. Because iWARP is layered on Internet Engineering Task Force (IETF)-standard congestion-aware protocols such as Transmission Control Protocol (TCP) and Stream Control Transmission Protocol (SCTP), it makes few requirements on the network, and can be successfully deployed in a broad range of environments. == History == In 2007, the IETF published five Request for Comments (RFCs) that define iWARP: RFC 5040 A Remote Direct Memory Access Protocol Specification is layered over Direct Data Placement Protocol (DDP). It defines how RDMA Send, Read, and Write operations are encoded using DDP into headers on the network. RFC 5041 Direct Data Placement over Reliable Transports is layered over MPA/TCP or SCTP. It defines how received data can be directly placed into an upper layer protocols receive buffer without intermediate buffers. RFC 5042 Direct Data Placement Protocol (DDP) / Remote Direct Memory Access Protocol (RDMAP) Security analyzes security issues related to iWARP DDP and RDMAP protocol layers. RFC 5043 Stream Control Transmission Protocol (SCTP) Direct Data Placement (DDP) Adaptation defines an adaptation layer that enables DDP over SCTP. RFC 5044 Marker PDU Aligned Framing for TCP Specification defines an adaptation layer that enables preservation of DDP-level protocol record boundaries layered over the TCP reliable connected byte stream. These RFCs are based on the RDMA Consortium's specifications for RDMA over TCP. The RDMA Consortium's specifications are influenced by earlier RDMA standards, including Virtual Interface Architecture (VIA) and InfiniBand (IB). Since 2007, the IETF has published three additional RFCs that maintain and extend iWARP: RFC 6580 IANA Registries for the Remote Direct Data Placement (RDDP) Protocols published in 2012 defines IANA registries for Remote Direct Data Placement (RDDP) error codes, operation codes, and function codes. RFC 6581 Enhanced Remote Direct Memory Access (RDMA) Connection Establishment published in 2011 fixes shortcomings with iWARP connection setup. RFC 7306 Remote Direct Memory Access (RDMA) Protocol Extensions published in 2014 extends RFC 5040 with atomic operations and RDMA Write with Immediate Data. == Protocol == The main component in the iWARP protocol is the Direct Data Placement Protocol (DDP), which permits the actual zero-copy transmission. DDP itself does not perform the transmission; the underlying protocol (TCP or SCTP) does. However, TCP does not respect message boundaries; it sends data as a sequence of bytes without regard to protocol data units (PDU). In this regard, DDP itself may be better suited for SCTP, and indeed the IETF proposed a standard RDMA over SCTP. To run DDP over TCP requires a tweak known as marker PDU aligned (MPA) framing to guarantee boundaries of messages. Furthermore, DDP is not intended to be accessed directly. Instead, a separate RDMA protocol (RDMAP) provides the services to read and write data. Therefore, the entire RDMA over TCP specification is really RDMAP over DDP over either MPA/TCP or SCTP. All of these protocols can be implemented in hardware. Unlike IB, iWARP only has reliable connected communication, as this is the only service that TCP and SCTP provide. The iWARP specification omits other features of IB, such as Send with Immediate Data operations. With RFC 7306, the IETF is working to reduce these omissions. == Implementation == Because a kernel implementation of the TCP stack can be seen as a bottleneck, the protocol is typically implemented in hardware RDMA network interface controllers (rNICs). As simple data losses are rare in tightly coupled network environments, the error-correction mechanisms of TCP may be performed by software while the more frequently performed communications are handled strictly by logic embedded on the rNIC. Similarly, connections are often established entirely by software and then handed off to the hardware. Furthermore, the handling of iWARP specific protocol details is typically isolated from the TCP implementation, allowing rNICs to be used for both as RDMA offload and TCP offload (in support of traditional sockets based TCP/IP applications). The portion of the hardware implementation used for implementing the TCP protocol is known as the TCP Offload Engine (TOE). TOE itself does not prevent copying on the reception side, and must be combined with RDMA hardware for zero-copy results. The RDMA / TCP specification is a set of different wire protocols intended to be implemented in hardware (though it seems feasible to emulate it in software for compatibility but without the performance benefits). == Interfaces == iWARP is a protocol, not an implementation, but defines protocol behavior in terms of the operations that are legal for the protocol, known as Verbs. As such, iWARP does not have any single standard programming interface. However, programming interfaces tend to very closely correspond to the Verbs. Several programmatic interfaces have been proposed, including OpenFabrics Verbs, Network Direct, uDAPL, kDAPL, IT-API, and RNICPI. Implementations of some of these interfaces are available for different platforms, including Windows and Linux. == Services available == Networking services implemented over iWARP include those offered in the OpenFabrics Enterprise Distribution (OFED) by the OpenFabrics Alliance for Linux operating systems, and by Microsoft Windows via Network Direct. NVMe over Fabrics (NVMEoF) iSCSI Extensions for RDMA (iSER) Server Message Block Direct (SMB Direct) Sockets Direct Protocol (SDP) SCSI RDMA Protocol (SRP) Network File System over RDMA (NFS over RDMA) GPUDirect

  Read more →