AI for Business

Explore the best AI for Business — independent reviews, comparisons, pricing and step-by-step how-to guides, curated by Aizhi.

  • Device-independent pixel

    Device-independent pixel

    A device-independent pixel (also: density-independent pixel, dip, dp) is a unit of length. A typical use is to allow mobile device software to scale the display of information and user interaction to different screen sizes. The abstraction allows an application to work in pixels as a measurement, while the underlying graphics system converts the abstract pixel measurements of the application into real pixel measurements appropriate to the particular device. For example, on the Android operating system a device-independent pixel is equivalent to one physical pixel on a 160 dpi screen, while the Windows Presentation Foundation specifies one device-independent pixel as equivalent to 1/96th of an inch. As dp is a physical unit it has an absolute value which can be measured in traditional units, e.g. for Android devices 1 dp equals 1/160 of inch or 0.15875 mm. While traditional pixels only refer to the display of information, device-independent pixels may also be used to measure user input such as input on a touch screen device.

    Read more →
  • SIPRNet

    SIPRNet

    The Secret Internet Protocol Router Network (SIPRNet) is "a system of interconnected computer networks used by the U.S. Department of Defense and the U.S. Department of State to transmit classified information (up to and including information classified SECRET) by packet switching over the 'completely secure' environment". It also provides services such as hypertext document access and electronic mail. SIPRNet is a component of the Defense Information Systems Network. Other components handle communications with other security needs, such as the NIPRNet, which is used for nonsecure communications, and the Joint Worldwide Intelligence Communications System (JWICS), which is used for Top Secret communications. == Access == According to the U.S. Department of State Web Development Handbook, domain structure and naming conventions are the same as for the open internet, except for the addition of a second-level domain, like, e.g., "sgov" between state and gov: openforum.state.sgov.gov. Files originating from SIPRNet are marked by a header tag "SIPDIS" (SIPrnet DIStribution). A corresponding second-level domain smil.mil exists for DoD users. Access is also available to a "...small pool of trusted allies, including Australia, Canada, the United Kingdom and New Zealand...". This group (including the US) is known as the Five Eyes. SIPRNet was one of the networks accessed by Chelsea Manning, convicted of leaking the video used in WikiLeaks' "Collateral Murder" release as well as the source of the US diplomatic cables published by WikiLeaks in November 2010. == Alternate names == SIPRNet and NIPRNet are referred to colloquially as SIPPERnet and NIPPERnet (or simply sipper and nipper), respectively.

    Read more →
  • Cambridge Analytica

    Cambridge Analytica

    Cambridge Analytica Ltd. (CA), previously known as SCL USA, was a British political consulting firm that came to prominence through the Facebook–Cambridge Analytica data scandal. It was founded in 2013, as a subsidiary of the private intelligence company and self-described "global election management agency" SCL Group by long-time SCL executives Nigel Oakes, Alexander Nix and Alexander Oakes, with Nix as CEO. Cambridge Analytica was hired by a variety of political actors, including the Trinidadian government in 2010 and the 2016 presidential campaigns of Ted Cruz and Donald Trump. The firm maintained offices in London, New York City, and Washington, D.C. The company closed operations in 2018 due to backlash from the scandal, although firms related to both Cambridge Analytica and its parent firm SCL still exist. == History == Cambridge Analytica was founded in 2013 as a subsidiary of the private intelligence company SCL Group, which describes itself as providing "data, analytics and strategy to governments and military organisations worldwide". The company was part of "an international web of companies" headed by the London-based SCL Group. Cambridge Analytica (SCL USA) was incorporated in January 2013 with its registered office being in Westferry Circus, London and consisting of just one staff member, director and CEO Alexander Nix (also appointed in January 2015). Nix was also the director of nine similar companies sharing the same registered offices in London, including Firecrest technologies, Emerdata and six SCL Group companies including "SCL elections limited". Nigel Oakes, known as the former boyfriend of Lady Helen Windsor, had founded the predecessor SCL Group in the 1990s, and in 2005 Oakes established SCL Group together with his brother Alexander Oakes and Alexander Nix; SCL Group was the parent company of Cambridge Analytica. Former Conservative minister and MP Sir Geoffrey Pattie was the founding chairman of SCL; Lord Ivar Mountbatten also joined Oakes as a director of the company. As a result of the Facebook–Cambridge Analytica data scandal, Nix was removed as CEO and replaced by Julian Wheatland before the company closed. Several of the company's executives were Old Etonians. The company's owners included several of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former British Conservative minister Jonathan Marland, Baron Marland and the family of American hedge fund manager Robert Mercer. The company combined misappropriation of digital assets, data mining, data brokerage, and data analysis with strategic communication during electoral processes. While its parent SCL had focused on influencing elections in developing countries since the 1990s, Cambridge Analytica focused more on the western world, including the United Kingdom and the United States; CEO Alexander Nix has said CA was involved in 44 U.S. political races in 2014. In 2015, CA performed data analysis services for Ted Cruz's presidential campaign. In 2016, CA worked for Donald Trump's presidential campaign as well as for Leave.EU (one of the organisations campaigning in the United Kingdom's referendum on European Union membership). CA's role in those campaigns has been controversial and is the subject of ongoing inquiries in both countries. Political scientists question CA's claims about the effectiveness of its methods of targeting voters. == Data scandal == In March 2018, media outlets broke news of Cambridge Analytica's business practices. The New York Times and The Observer reported that the company had acquired and used personal data about Facebook users from an external researcher who had told Facebook he was collecting it for academic purposes. Shortly afterwards, Channel 4 News aired undercover investigative videos showing Nix boasting about using prostitutes, bribery sting operations, and honey traps to discredit politicians on whom it had conducted opposition research, and saying that the company "ran all of (Donald Trump's) digital campaign". In response to the media reports, the Information Commissioner's Office (ICO) of the UK pursued a warrant to search the company's servers. Facebook banned Cambridge Analytica from advertising on its platform, saying that it had been deceived. On 23 March 2018, the British High Court granted the ICO a warrant to search Cambridge Analytica's London offices. As a result, Nix was suspended as CEO, and replaced by Julian Wheatland. The personal data of up to 87 million Facebook users were acquired via the 270,000 Facebook users who used a Facebook app created by Aleksandr Kogan called "This Is Your Digital Life". This was a personality profiling app and asked simple personality questions similar to other Facebook quizzes. Kogan was a scientist and psychologist, also being an employed lecturer for the University of Cambridge from 2012 to 2018. Alexander Nix claimed they had close to five thousand data points on each person who participated. They also gathered information through other data brokers ending with them acquiring millions of data points from American citizens. Kogan's app exploited a feature of Facebook's Graph API (version 1.0), which permitted any third-party app to access not only the app user's data, but also the full profile data of all of that user's Facebook friends, without those friends' knowledge or consent. This platform-wide design was available to all developers and was used by tens of thousands of apps; Facebook CEO Mark Zuckerberg later told the House Energy and Commerce Committee that the company was auditing "tens of thousands" of apps that had had access to large amounts of user data. Because the average Facebook user at the time had approximately 300 friends, the 270,000 users who installed Kogan's app yielded data on up to 87 million people. Facebook deprecated the friends-data API in April 2014 and shut it down entirely in April 2015, but data already collected by apps remained in developers' possession. Kogan passed this data to Cambridge Analytica, breaching Facebook's terms of service. On 1 May 2018, Cambridge Analytica and its parent company SCL filed for insolvency proceedings and closed operations. Alexander Tayler, a former director for Cambridge Analytica, was appointed director of Emerdata on 28 March 2018. Rebekah Mercer, Jennifer Mercer, Alexander Nix and Johnson Chun Shun Ko, who has links to American businessman Erik Prince, are in leadership positions at Emerdata. The Russo brothers are producing an upcoming film on Cambridge Analytica. In 2019 the Federal Trade Commission filed an administrative complaint against Cambridge Analytica for misuse of data. In 2020, the British Information Commissioner's Office closed a three-year inquiry into the company, concluded that Cambridge Analytica was "not involved" in the 2016 Brexit referendum and found no additional evidence for Russia's alleged interference during the campaign. US sensitive polling and election data, however, were passed to Russian Intelligence via a Cambridge Analytica contractor Sam Patten, Trump campaign manager Paul Manafort, and Russian agent Konstantin Kilimnik, who was indicted during the affair. Publicly, parent company SCL Group called itself a "global election management agency", Politico reported it was known for involvement "in military disinformation campaigns to social media branding and voter targeting". SCL gained work on a large number of campaigns for the US and UK governments' war on terror advancing their model of behavioral conflict during the 2000s. SCL's involvement in the political world has been primarily in the developing world where it has been used by the military and politicians to study and manipulate public opinion and political will. Slate writer Sharon Weinberger compared one of SCL's hypothetical test scenarios to fomenting a coup. Among the investors in Cambridge Analytica were some of the Conservative Party's largest donors such as billionaire Vincent Tchenguiz, former Conservative minister Jonathan Marland, Baron Marland, Roger Gabb, the family of American hedge fund manager Robert Mercer, and Steve Bannon. A minimum of 15 million dollars has been invested into the company by Mercer, according to The New York Times. Bannon's stake in the company was estimated at 1 to 5 million dollars, but he divested his holdings in April 2017 as required by his role as White House Chief Strategist. In March 2018, Jennifer Mercer and Rebekah Mercer became directors of Emerdata limited. In March 2018 it became public by Christopher Wylie, that Cambridge Analytica's first activities were founded on a data set, which its parent company SCL bought 2014 from a company named Global Science Research founded by Aleksandr Kogan and his team present across the world who worked as a psychologist at Cambridge. During Boris Johnson's tenure as foreign secretary, the Foreign Office sought advice from Cambridge Analytica and Boris Johnson had a meeting with Alexander N

    Read more →
  • Social media as a news source

    Social media as a news source

    Social media as a news source is defined as the use of online social media platforms such as Instagram, TikTok, and Facebook rather than the use of traditional media platforms like the newspaper or live TV to obtain news. Television had just begun to turn a nation of people who once listened to media content into watchers of media content between the 1950s and the 1980s when the popularity of social media had also begun creating a nation of media content creators. Almost half of Americans use social media as a news source, according to the Pew Research Center. As social media's role in news consumption grows, questions have emerged about its impact on knowledge, the formation of echo chambers, and the effectiveness of fact-checking efforts in combating misinformation. Social media platforms allow user-generated content and sharing content within one's own virtual network. Using social media as a news source allows users to engage with news in a variety of ways including: Consuming and discovering news Sharing or reposting news Posting one's own photos, videos, or reports of news (i.e., engage in citizen or participatory journalism) Commenting on news posts Using social media as a news source has become an increasingly popular way for people of all age groups to obtain current and important information. Just like many other new forms of technology there are going to be pros and cons. There are ways that social media positively affects the world of news and journalism but it is important to acknowledge that there are also ways in which social media has a negative effect on the news. With this accessibility, people now have more ways to consume false news, biased news, and even disturbing content. In 2019, the Pew Research Center created a poll that reported Americans are wary about the ways that social media sites share news and certain content. This wariness of accuracy grew as awareness that social media sites could be exploited by bad actors who concoct false narratives and fake news. == Relationship to traditional news sources == Unlike traditional news platforms such as newspapers and news shows, social media platforms allow people without professional journalistic backgrounds to create news and cover events that news agencies might not cover. Social media users may read a set of news that differs slightly from what newspaper editors prioritize in the print press. A 2019 study found that Facebook and Twitter users are more likely to share politics, public affairs, and visual media news. Typically social media users circulate more towards posting about negative news. A study of tweets found that while optimistic-sounding and neutral-sounding tweets were equally likely to express certainty or uncertainty, the pessimistic tweets were nearly twice as likely to appear certain of an outcome than uncertain. These results could imply that posts of a more pessimistic nature that are also written with an air of certainty are more likely to be shared or otherwise permeate groups on Twitter. A similar bias towards negativity has developed on Facebook, where internal memos revealed that an algorithm built to promote "meaningful social interaction" actually incentivized publishers to promote negative and sensational news. Biases towards negativity need to be considered when the utility of new media is addressed, as the potential for human opinion to overemphasize any particular news story is greater despite general improvement. In order to compete in this rapidly changing technological environment, there has been an upheaval of traditional news sources onto online spaces. The production and circulation of newspaper prints have continued to globally decline in accordance with the increasing presence of news outlets on social media. Prominent platforms such as Twitter and Facebook have been key in engaging users through the integration of journalistic news into their newsfeeds. This feature has now become a foundational part of these apps' interfaces. Social media incentivizes both legacy news brands and individual professional journalists to share their reporting and interact with audiences on social platforms to boost engagement. However, most people who consume news on social media report that accessing news is not their main motivation for being on social media, but rather, they see and consume news incidentally. Nonetheless, informational interviews reveal that these consumers rely on being informed through social media. Some news consumers attest that a news brand's participation in social media does not improve their trust in the brand and that more in-depth reporting and more transparency about biases would improve trust instead. == Use as a news source == Globally, data from 2020 shows that over 70% of adult participants from Kenya, South Africa, Chile, Bulgaria, Greece, and Argentina utilized social media for news while those from France, the UK, the Netherlands, Germany, and Japan were reportedly less than 40 percent. According to the Pew Research Center, 20% of adults in the United States in 2018 said they get their news from social media "often," compared to 16% who said they often get news from print newspapers, 26% who often get it from the radio, 33% who often get it from news websites, and 49% who often get it from TV. The same survey found that social media was the most popular way for American adults age 18–29 to get news, the second-to-last most popular way for Americans age 20–49 to get news, and the least popular way for American adults age 50-64 and 65+ to get the news. In 2019, the Pew Research Center found that over half of Americans (54%) either got their news "sometimes" or "often" from social media, and Facebook was the most popular social media site where American adults got their news. However, at least 50% off all respondents reported that the following were either a "very big problem" or a "moderately big problem" for getting news on social media: One-sided news (83%) Inaccurate news (81%) Censorship of the news (69%) Uncivil discussions about the news (69%) Harassment of journalists (57%) News organizations or personalities being banned (53%) Violent or disturbing news images or videos (51%) In a later survey from the same year, the Pew Research Center reported that 18% of American adults reported that the most common way they get news about politics and the election was from social media. Additional source information shows that from politics and the United States presidential election in 2016, the popularity of fake news had grown to global attention. With this information, the study explains that more than 60 percent of adults receive their news from social media, the most popular being Facebook. With the increase of fake news, and the large amount of adult participation on these social media sites, it made it much harder for those who were searching for news to find a source that they could find credible. Another study found that adult participants found their own friends on Facebook to be a more reliable source of information online compared to a professional news organization. Although, when news was posted by a news organization online, they were then found more reliable compared to when they are shared by their online friends. Showing that adult participants found that the news that was only posted on Facebook and social media was much more credible to them than compared to other forms of information spreading. The study further states that these outcomes have the potential explanation that the topic of the news article played a part in the ways they were affected. This could have affected the way adult participants interacted with the different news sources, such as their online friends compared to a news organization, prominently because depending on the story, they want to have the correct information about the news from the most credible source. === By young people === Social media platforms are some of the most easily accessible forms of news and with the growing generations, the technology is only going to grow. With that, the use of social media in younger generations is also going to grow alongside it. Technology in the hands of young kids can be a concern moving into the future. Globally, there is evidence that through social media, youth have become more directly involved in protests, social campaigns and generally, in the sharing of news across multiple platforms. The number of people who use social media platforms such as Twitter, Facebook, Instagram, or Snapchat as ways to seek information has increased significantly in recent years especially for people who are part of the younger generation.TikTok is a rapidly expanding platform that young adults can use to find news content on social media. TikTok is one of the sites that young adults and teens utilize to get news about trending themes and controversial topics. The younger generation accepts without hesitation the information that thei

    Read more →
  • Semantic analysis (machine learning)

    Semantic analysis (machine learning)

    In machine learning, semantic analysis of a text corpus is the task of building structures that approximate concepts from a large set of documents. It generally does not involve prior semantic understanding of the documents. Semantic analysis strategies include: Metalanguages based on first-order logic, which can analyze the speech of humans. Understanding the semantics of a text is symbol grounding: if language is grounded, it is equal to recognizing a machine-readable meaning. For the restricted domain of spatial analysis, a computer-based language understanding system was demonstrated. Latent semantic analysis (LSA), a class of techniques where documents are represented as vectors in a term space. A prominent example is probabilistic latent semantic analysis (PLSA). Latent Dirichlet allocation, which involves attributing document terms to topics. n-grams and hidden Markov models, which work by representing the term stream as a Markov chain, in which each term is derived from preceding terms. == Stochastic semantic analysis ==

    Read more →
  • Viber

    Viber

    Rakuten Viber, commonly known as Viber, is a cross-platform voice over IP (VoIP) and instant messaging (IM) software application owned by the Japanese technology company Rakuten Group. The service is available as freeware for Android, iOS, Microsoft Windows, macOS and Linux. Users are registered and identified through a mobile phone number, although the service can also be accessed on desktop platforms without mobile connectivity. In addition to instant messaging, the platform allows users to exchange media such as images, videos and files, and provides a paid international calling service called Viber Out. The software was launched in 2010 by the company Viber Media, founded by Talmon Marco and Igor Magazinnik. Rakuten acquired Viber Media in 2014 and later renamed the company Rakuten Viber. The company is headquartered in Cyprus and maintains offices in London, Manila, Paris, San Francisco, Singapore, Tokyo and Beijing. == History == === Founding (2010) === Viber Media was founded in Tel Aviv, Israel, in 2010 by Talmon Marco and Igor Magazinnik. Marco and Magazinnik are also co-founders of the peer-to-peer media and file-sharing client iMesh. The company was run from Israel and was registered in Cyprus. Sani Maroli and Ofer Smocha soon joined the company as well. Marco said Viber allows instant calling and synchronization with contacts because the ID is the user's cell number. In its early days, Viber relied on a patchwork of outsourcing partners from different countries, commissioning specific solutions from external vendors — including teams based in Cyprus and Belarus. According to the company's statements, development of Viber's core functionality historically originated from its Tel Aviv office — a testament to its roots — even though the legal entity was registered elsewhere. === Early monetisation (2011) === In its first two years of availability, Viber did not generate revenues. It began doing so in 2013, via user payments for Viber Out voice calling and the Viber graphical messaging "sticker market". The company was originally funded by individual investors, described by Marco as "friends and family". They invested $20 million in the company, which had 120 employees as of May 2013. On 24 July 2013, Viber's support system was defaced by the Syrian Electronic Army. According to Viber, no sensitive user information was accessed. By the time Rakuten came forward with its acquisition deal in 2014, Viber had already stopped working with external vendors, choosing instead to consolidate development under its own offices. === Rakuten acquires Viber (2014) === On 13 February 2014, Rakuten announced they had acquired Viber Media for $900 million, and since then Viber has been owned by Rakuten, Inc., an e-commerce conglomerate headquartered in Tokyo. The sale of Viber earned the Shabtai family (Benny, his brother Gilad, and Gilad's son Ofer) some $500 million from their 55.2% stake in the company. At that sale price, the founders each realized over 30 times return on their investments. Later that year, the company established a UK presence with the incorporation of Viber UK Limited in London. Djamel Agaoua became Viber Media CEO in February 2017, replacing co-founder Marco who left in 2015. In July 2017 the corporate name of Viber Media was changed to Rakuten Viber and a new wordmark logo was introduced. Its legal name remains Viber Media, S.à r.l. based in Luxembourg. === Post-acquisition === In August 2015 Viber opened a regional office for Central and Eastern Europe in Sofia to support growth in the region. In 2017, Rakuten Viber and the World Wildlife Fund engaged in a commercial transaction aimed at raising awareness and protecting wildlife. After first using Viber to spread its message in June 2020, the International Federation of the Red Cross launched an official chatbot and community on the messaging app to combat the spread of false information, which they termed an infodemic, about COVID-19. The chatbot is still active as of June 2022, with over 1.4 million subscribers. In 2020, Rakuten Viber and the World Health Organization (the WHO) engaged in a commercial transaction for a chatbot to inform users of issues such as women's health. and an anti-smoking campaign. In the wake of the July–August 2020 Belarusian election protests, to avoid sanctions and harassment from monopolies the company closed its office in Minsk. In 2022, Ofir Eyal became Viber CEO, replacing Djamel Agaoua. Eyal is a Viber veteran; he worked as Vice President of Product in 2014 before his promotion to Chief Operating Officer in 2019. Shortly after the appointment of a new CEO, Viber continued its international expansion. In March 2022, Rakuten announced the opening of a development center in Tbilisi, Georgia, intended to support work on mobile applications and technology projects in the region. In July 2022, Rakuten Viber partnered with Rapyd to launch instant cross-border P2P payments. The company launched payments on the Viber app first in Greece and Germany, and then in other countries. In August, Mineski teamed up with Viber to develop a social minigame platform that can play off Viber's application. In May 2022, Rakuten Viber launched the premium chat service Viber Plus that offers exclusive features, including sticker market privileges, ad-free use, priority Viber support, exclusive badge, unique Viber icon, large file sharing, and more. In 2022, Viber joined the European Union’s Code of Conduct on countering illegal hate speech online. As part of this framework, the company undertook to review reported content and remove material identified as hate speech in accordance with the Code and its platform rules. In January 2024 Rakuten (the company behind Viber) established an office in Kyiv to bring together engineering and marketing departments. Alongside launching its Kyiv office the company joined Diia.City as a resident. Subsequently in October 2024 Rakuten Viber inaugurated an office in Manila to broaden its operations, in the Philippines. The company’s legal entity remains Viber Media S.à r.l., registered in Luxembourg. Viber’s engineering work has been carried out across multiple countries and through external partners, including outsourcing and near-shore vendors. As a result, its development operations are distributed internationally rather than concentrated in a single location. In December 2024, Viber was blocked in Russia. Roskomnadzor announced the nationwide blocking of the messaging app due to non-compliance with local legal requirements. == Security audit == On 4 November 2014, Viber scored 1 out of 7 points on the Electronic Frontier Foundation's "Secure Messaging Scorecard". Viber received a point for encryption during transit but lost points because communications were not encrypted with keys that the provider did not have access to (i.e. the communications were not end-to-end encrypted), users could not verify contacts' identities, past messages were not secure if the encryption keys were stolen (i.e. the service did not provide forward secrecy), the code was not open to independent review (i.e. the code was not open-source), the security design was not properly documented, and there had not been a recent independent security audit. On 14 November 2014, the EFF changed Viber's score to 2 out of 7 after it had received an external security audit from Ernst & Young's Advanced Security Centre. On 19 April 2016, with the announcement of Viber version 6.0, Rakuten added end-to-end encryption to their service. The company said that the encryption protocol had only been audited internally, and promised to commission external audits "in the coming weeks". In May 2016, Viber published an overview of their encryption protocol, saying that it is a custom implementation that "uses the same concepts" as the Signal Protocol. In 2022, Rakuten Viber won a Security Award, by test.de, a tech firm based in Germany where there are over 3 million Viber users. In 2024, Rakuten Viber received SOC certification following an audit conducted by Ernst & Young. The certification relates to the company’s controls for data protection and information security. == Market share == As of December 2016, Viber had 800 million registered users. According to Statista, there are 260 million monthly active users as of January 2019. The Viber messenger is very popular in the Philippines, Greece, Eastern Europe, Russia, the Middle East, and some Asian markets. India was the largest market for Viber as of December 2014 with 33 million registered users, the fifth most popular instant messenger in the country. At the same time there were 30 million users in the United States, 28 million in Russia and 18 million in Brazil. Viber is particularly popular in Eastern Europe, being the most downloaded messaging app on Android in Belarus, Moldova and Ukraine as of 2016. It is also popular in Iraq, Libya and Nepal. Viber is translated in 44 languages and used in more than 190 co

    Read more →
  • Back-Up Interceptor Control

    Back-Up Interceptor Control

    Backup Interceptor Control (BUIC, ) was the Electronic Systems Division 416M System to backup the SAGE 416L System in the United States and Canada. BUIC deployed Cold War command, control, and coordination systems to SAGE radar stations to create dispersed NORAD Control Centers. == Background == Prior to the SAGE Direction Centers becoming operational, the USAF deployed data link systems at NORAD Control Centers with ground computers for controlling crewed interceptors. After SAGE IBM AN/FSQ-7 Combat Direction Centrals became operational and the Super Combat Centers with improved (digital) computers were cancelled, a backup to SAGE was planned in the event the above-ground SAGE Air Defense Direction Center failed. == General Electric AN/GPA-37 Course Directing Group == BUIC began with deployment of General Electric AN/GPA-37 Course Directing Groups to several Long Range Radar stations. Units designated included the "U.S. Air Force 858th Air Defense Group (BUIC) [which became] a permanent operating facility" at Naval Air Station Fallon in Nevada. == BUIC II == BUIC II was used to command and control sites using the Burroughs AN/GSA-51 Radar Course Directing Group. North Truro AFS became the first ADC installation configured for BUIC II. == BUIC III == The AN/GYK-19 (initially AN/GSA-51A) was an upgraded version of the BUIC II system designated AN/GSA-51A and required a larger building than the AN/GSA-51. The first BUIC III site was Fort Fisher AFS, and Air Defense Command's was first installed at Fort Fisher Air Force Station, North Carolina. Although more advanced systems were contemplated, the final design of the BUIC III system was an upgraded version of the BUIC II with around twice the performance. == Closure and upgrade == In 1972, the USAF decided to shut down most of the BUIC sites; most of the sites mothballed by 1974, except for the BUIC III site at Tyndall Air Force Base. In Canada the BUIC site at Senneterre was shut down, but St Margarets remained open. The remaining sites were closed between 1983-1984 when SAGE was replaced by the Joint Surveillance System. The AN/FYQ-47 Common Digitizer for the Joint Surveillance System, and the Radar Video Data Processor (RVDP) was a combined system for the Air Force and Federal Aviation Administration (FAA), it replaced the SAGE Burroughs AN/FST-2 Coordinate Data Transmitting Sets.

    Read more →
  • Locally recoverable code

    Locally recoverable code

    Locally recoverable codes are a family of error correction codes that were introduced first by D. S. Papailiopoulos and A. G. Dimakis and have been widely studied in information theory due to their applications related to distributive and cloud storage systems. An [ n , k , d , r ] q {\displaystyle [n,k,d,r]_{q}} LRC is an [ n , k , d ] q {\displaystyle [n,k,d]_{q}} linear code such that there is a function f i {\displaystyle f_{i}} that takes as input i {\displaystyle i} and a set of r {\displaystyle r} other coordinates of a codeword c = ( c 1 , … , c n ) ∈ C {\displaystyle c=(c_{1},\ldots ,c_{n})\in C} different from c i {\displaystyle c_{i}} , and outputs c i {\displaystyle c_{i}} . == Overview == Erasure-correcting codes, or simply erasure codes, for distributed and cloud storage systems, are becoming more and more popular as a result of the present spike in demand for cloud computing and storage services. This has inspired researchers in the fields of information and coding theory to investigate new facets of codes that are specifically suited for use with storage systems. It is well-known that LRC is a code that needs only a limited set of other symbols to be accessed in order to restore every symbol in a codeword. This idea is very important for distributed and cloud storage systems since the most common error case is when one storage node fails (erasure). The main objective is to recover as much data as possible from the fewest additional storage nodes in order to restore the node. Hence, Locally Recoverable Codes are crucial for such systems. The following definition of the LRC follows from the description above: an [ n , k , r ] {\displaystyle [n,k,r]} -Locally Recoverable Code (LRC) of length n {\displaystyle n} is a code that produces an n {\displaystyle n} -symbol codeword from k {\displaystyle k} information symbols, and for any symbol of the codeword, there exist at most r {\displaystyle r} other symbols such that the value of the symbol can be recovered from them. The locality parameter satisfies 1 ≤ r ≤ k {\displaystyle 1\leq r\leq k} because the entire codeword can be found by accessing k {\displaystyle k} symbols other than the erased symbol. Furthermore, Locally Recoverable Codes, having the minimum distance d {\displaystyle d} , can recover d − 1 {\displaystyle d-1} erasures. == Definition == Let C {\displaystyle C} be a [ n , k , d ] q {\displaystyle [n,k,d]_{q}} linear code. For i ∈ { 1 , … , n } {\displaystyle i\in \{1,\ldots ,n\}} , let us denote by r i {\displaystyle r_{i}} the minimum number of other coordinates we have to look at to recover an erasure in coordinate i {\displaystyle i} . The number r i {\displaystyle r_{i}} is said to be the locality of the i {\displaystyle i} -th coordinate of the code. The locality of the code is defined as An [ n , k , d , r ] q {\displaystyle [n,k,d,r]_{q}} locally recoverable code (LRC) is an [ n , k , d ] q {\displaystyle [n,k,d]_{q}} linear code C ∈ F q n {\displaystyle C\in \mathbb {F} _{q}^{n}} with locality r {\displaystyle r} . Let C {\displaystyle C} be an [ n , k , d ] q {\displaystyle [n,k,d]_{q}} -locally recoverable code. Then an erased component can be recovered linearly, i.e. for every i ∈ { 1 , … , n } {\displaystyle i\in \{1,\ldots ,n\}} , the space of linear equations of the code contains elements of the form x i = f ( x i 1 , … , x i r ) {\displaystyle x_{i}=f(x_{i_{1}},\ldots ,x_{i_{r}})} , where i j ≠ i {\displaystyle i_{j}\neq i} . == Optimal locally recoverable codes == Theorem Let n = ( r + 1 ) s {\displaystyle n=(r+1)s} and let C {\displaystyle C} be an [ n , k , d ] q {\displaystyle [n,k,d]_{q}} -locally recoverable code having s {\displaystyle s} disjoint locality sets of size r + 1 {\displaystyle r+1} . Then An [ n , k , d , r ] q {\displaystyle [n,k,d,r]_{q}} -LRC C {\displaystyle C} is said to be optimal if the minimum distance of C {\displaystyle C} satisfies == Tamo–Barg codes == Let f ∈ F q [ x ] {\displaystyle f\in \mathbb {F} _{q}[x]} be a polynomial and let ℓ {\displaystyle \ell } be a positive integer. Then f {\displaystyle f} is said to be ( r {\displaystyle r} , ℓ {\displaystyle \ell } )-good if • f {\displaystyle f} has degree r + 1 {\displaystyle r+1} , • there exist distinct subsets A 1 , … , A ℓ {\displaystyle A_{1},\ldots ,A_{\ell }} of F q {\displaystyle \mathbb {F} _{q}} such that – for any i ∈ { 1 , … , ℓ } {\displaystyle i\in \{1,\ldots ,\ell \}} , f ( A i ) = { t i } {\displaystyle f(A_{i})=\{t_{i}\}} for some t i ∈ F q {\displaystyle t_{i}\in \mathbb {F} _{q}} , i.e., f {\displaystyle f} is constant on A i {\displaystyle A_{i}} , – # A i = r + 1 {\displaystyle \#A_{i}=r+1} , – A i ∩ A j = ∅ {\displaystyle A_{i}\cap A_{j}=\varnothing } for any i ≠ j {\displaystyle i\neq j} . We say that { A 1 , … , A ℓ {\displaystyle A_{1},\ldots ,A_{\ell }} } is a splitting covering for f {\displaystyle f} . === Tamo–Barg construction === The Tamo–Barg construction utilizes good polynomials. • Suppose that a ( r , ℓ ) {\displaystyle (r,\ell )} -good polynomial f ( x ) {\displaystyle f(x)} over F q {\displaystyle \mathbb {F} _{q}} is given with splitting covering i ∈ { 1 , … , ℓ } {\displaystyle i\in \{1,\ldots ,\ell \}} . • Let s ≤ ℓ − 1 {\displaystyle s\leq \ell -1} be a positive integer. • Consider the following F q {\displaystyle \mathbb {F} _{q}} -vector space of polynomials V = { ∑ i = 0 s g i ( x ) f ( x ) i : deg ⁡ ( g i ( x ) ) ≤ deg ⁡ ( f ( x ) ) − 2 } . {\displaystyle V=\left\{\sum _{i=0}^{s}g_{i}(x)f(x)^{i}:\deg(g_{i}(x))\leq \deg(f(x))-2\right\}.} • Let T = ⋃ i = 1 ℓ A i {\textstyle T=\bigcup _{i=1}^{\ell }A_{i}} . • The code { ev T ⁡ ( g ) : g ∈ V } {\displaystyle \{\operatorname {ev} _{T}(g):g\in V\}} is an ( ( r + 1 ) ℓ , ( s + 1 ) r , d , r ) {\displaystyle ((r+1)\ell ,(s+1)r,d,r)} -optimal locally coverable code, where ev T {\displaystyle \operatorname {ev} _{T}} denotes evaluation of g {\displaystyle g} at all points in the set T {\displaystyle T} . === Parameters of Tamo–Barg codes === • Length. The length is the number of evaluation points. Because the sets A i {\displaystyle A_{i}} are disjoint for i ∈ { 1 , … , ℓ } {\displaystyle i\in \{1,\ldots ,\ell \}} , the length of the code is | T | = ( r + 1 ) ℓ {\displaystyle |T|=(r+1)\ell } . • Dimension. The dimension of the code is ( s + 1 ) r {\displaystyle (s+1)r} , for s {\displaystyle s} ≤ ℓ − 1 {\displaystyle \ell -1} , as each g i {\displaystyle g_{i}} has degree at most deg ⁡ ( f ( x ) ) − 2 {\displaystyle \deg(f(x))-2} , covering a vector space of dimension deg ⁡ ( f ( x ) ) − 1 = r {\displaystyle \deg(f(x))-1=r} , and by the construction of V {\displaystyle V} , there are s + 1 {\displaystyle s+1} distinct g i {\displaystyle g_{i}} . • Distance. The distance is given by the fact that V ⊆ F q [ x ] ≤ k {\displaystyle V\subseteq \mathbb {F} _{q}[x]_{\leq k}} , where k = r + 1 − 2 + s ( r + 1 ) {\displaystyle k=r+1-2+s(r+1)} , and the obtained code is the Reed-Solomon code of degree at most k {\displaystyle k} , so the minimum distance equals ( r + 1 ) ℓ − ( ( r + 1 ) − 2 + s ( r + 1 ) ) {\displaystyle (r+1)\ell -((r+1)-2+s(r+1))} . • Locality. After the erasure of the single component, the evaluation at a i ∈ A i {\displaystyle a_{i}\in A_{i}} , where | A i | = r + 1 {\displaystyle |A_{i}|=r+1} , is unknown, but the evaluations for all other a ∈ A i {\displaystyle a\in A_{i}} are known, so at most r {\displaystyle r} evaluations are needed to uniquely determine the erased component, which gives us the locality of r {\displaystyle r} . To see this, g {\displaystyle g} restricted to A j {\displaystyle A_{j}} can be described by a polynomial h {\displaystyle h} of degree at most deg ⁡ ( f ( x ) ) − 2 = r + 1 − 2 = r − 1 {\displaystyle \deg(f(x))-2=r+1-2=r-1} thanks to the form of the elements in V {\displaystyle V} (i.e., thanks to the fact that f {\displaystyle f} is constant on A j {\displaystyle A_{j}} , and the g i {\displaystyle g_{i}} 's have degree at most deg ⁡ ( f ( x ) ) − 2 {\displaystyle \deg(f(x))-2} ). On the other hand | A j ∖ { a j } | = r {\displaystyle |A_{j}\backslash \{a_{j}\}|=r} , and r {\displaystyle r} evaluations uniquely determine a polynomial of degree r − 1 {\displaystyle r-1} . Therefore h {\displaystyle h} can be constructed and evaluated at a j {\displaystyle a_{j}} to recover g ( a j ) {\displaystyle g(a_{j})} . === Example of Tamo–Barg construction === We will use x 5 ∈ F 41 [ x ] {\displaystyle x^{5}\in \mathbb {F} _{41}[x]} to construct [ 15 , 8 , 6 , 4 ] {\displaystyle [15,8,6,4]} -LRC. Notice that the degree of this polynomial is 5, and it is constant on A i {\displaystyle A_{i}} for i ∈ { 1 , … , 8 } {\displaystyle i\in \{1,\ldots ,8\}} , where A 1 = { 1 , 10 , 16 , 18 , 37 } {\displaystyle A_{1}=\{1,10,16,18,37\}} , A 2 = 2 A 1 {\displaystyle A_{2}=2A_{1}} , A 3 = 3 A 1 {\displaystyle A_{3}=3A_{1}} , A 4 = 4 A 1 {\displaystyle A_{4}=4A_{1}} , A 5 = 5 A 1 {\displaystyle A_{5}=5A_{1}} , A 6 = 6 A 1 {\displaystyle A_{6}=6A_{1}}

    Read more →
  • Semantic folding

    Semantic folding

    Semantic folding theory describes a procedure for encoding the semantics of natural language text in a semantically grounded binary representation. This approach provides a framework for modelling how language data is processed by the neocortex. == Theory == Semantic folding theory draws inspiration from Douglas R. Hofstadter's Analogy as the Core of Cognition which suggests that the brain makes sense of the world by identifying and applying analogies. The theory hypothesises that semantic data must therefore be introduced to the neocortex in such a form as to allow the application of a similarity measure and offers, as a solution, the sparse binary vector employing a two-dimensional topographic semantic space as a distributional reference frame. The theory builds on the computational theory of the human cortex known as hierarchical temporal memory (HTM), and positions itself as a complementary theory for the representation of language semantics. A particular strength claimed by this approach is that the resulting binary representation enables complex semantic operations to be performed simply and efficiently at the most basic computational level. == Two-dimensional semantic space == Analogous to the structure of the neocortex, Semantic Folding theory posits the implementation of a semantic space as a two-dimensional grid. This grid is populated by context-vectors in such a way as to place similar context-vectors closer to each other, for instance, by using competitive learning principles. This vector space model is presented in the theory as an equivalence to the well known word space model described in the information retrieval literature. Given a semantic space (implemented as described above) a word-vector can be obtained for any given word Y by employing the following algorithm: For each position X in the semantic map (where X represents cartesian coordinates) if the word Y is contained in the context-vector at position X then add 1 to the corresponding position in the word-vector for Y else add 0 to the corresponding position in the word-vector for Y The result of this process will be a word-vector containing all the contexts in which the word Y appears and will therefore be representative of the semantics of that word in the semantic space. It can be seen that the resulting word-vector is also in a sparse distributed representation (SDR) format [Schütze, 1993] & [Sahlgreen, 2006]. Some properties of word-SDRs that are of particular interest with respect to computational semantics are: high noise resistance: As a result of similar contexts being placed closer together in the underlying map, word-SDRs are highly tolerant of false or shifted "bits". boolean logic: It is possible to manipulate word-SDRs in a meaningful way using boolean (OR, AND, exclusive-OR) and/or arithmetical (SUBtract) functions . sub-sampling: Word-SDRs can be sub-sampled to a high degree without any appreciable loss of semantic information. topological two-dimensional representation: The SDR representation maintains the topological distribution of the underlying map therefore words with similar meanings will have similar word-vectors. This suggests that a variety of measures can be applied to the calculation of semantic similarity, from a simple overlap of vector elements, to a range of distance measures such as: Euclidean distance, Hamming distance, Jaccard distance, cosine similarity, Levenshtein distance, Sørensen-Dice index, etc. == Semantic spaces == Semantic spaces in the natural language domain aim to create representations of natural language that are capable of capturing meaning. The original motivation for semantic spaces stems from two core challenges of natural language: Vocabulary mismatch (the fact that the same meaning can be expressed in many ways) and ambiguity of natural language (the fact that the same term can have several meanings). The application of semantic spaces in natural language processing (NLP) aims at overcoming limitations of rule-based or model-based approaches operating on the keyword level. The main drawback with these approaches is their brittleness, and the large manual effort required to create either rule-based NLP systems or training corpora for model learning. Rule-based and machine learning-based models are fixed on the keyword level and break down if the vocabulary differs from that defined in the rules or from the training material used for the statistical models. Research in semantic spaces dates back more than 20 years. In 1996, two papers were published that raised a lot of attention around the general idea of creating semantic spaces: latent semantic analysis from Microsoft and Hyperspace Analogue to Language from the University of California. However, their adoption was limited by the large computational effort required to construct and use those semantic spaces. A breakthrough with regard to the accuracy of modelling associative relations between words (e.g. "spider-web", "lighter-cigarette", as opposed to synonymous relations such as "whale-dolphin", "astronaut-driver") was achieved by explicit semantic analysis (ESA) in 2007. ESA was a novel (non-machine learning) based approach that represented words in the form of vectors with 100,000 dimensions (where each dimension represents an Article in Wikipedia). However practical applications of the approach are limited due to the large number of required dimensions in the vectors. More recently, advances in neural networking techniques in combination with other new approaches (tensors) led to a host of new recent developments: Word2vec from Google and GloVe from Stanford University. Semantic folding represents a novel, biologically inspired approach to semantic spaces where each word is represented as a sparse binary vector with 16,000 dimensions (a semantic fingerprint) in a 2D semantic map (the semantic universe). Sparse binary representation are advantageous in terms of computational efficiency, and allow for the storage of very large numbers of possible patterns. == Visualization == The topological distribution over a two-dimensional grid (outlined above) lends itself to a bitmap type visualization of the semantics of any word or text, where each active semantic feature can be displayed as e.g. a pixel. As can be seen in the images shown here, this representation allows for a direct visual comparison of the semantics of two (or more) linguistic items. Image 1 clearly demonstrates that the two disparate terms "dog" and "car" have, as expected, very obviously different semantics. Image 2 shows that only one of the meaning contexts of "jaguar", that of "Jaguar" the car, overlaps with the meaning of Porsche (indicating partial similarity). Other meaning contexts of "jaguar" e.g. "jaguar" the animal clearly have different non-overlapping contexts. The visualization of semantic similarity using Semantic Folding bears a strong resemblance to the fMRI images produced in a research study conducted by A.G. Huth et al., where it is claimed that words are grouped in the brain by meaning. voxels, little volume segments of the brain, were found to follow a pattern were semantic information is represented along the boundary of the visual cortex with visual and linguistic categories represented on posterior and anterior side respectively.

    Read more →
  • Intranet

    Intranet

    An intranet is a computer network for sharing information, easier communication, collaboration tools, operational systems, and other computing services within an organization, usually to the exclusion of access by outsiders. The term is used in contrast to public networks, such as the Internet, but uses the same technology based on the Internet protocol suite. An organization-wide intranet can constitute a focal point of internal communication and collaboration, and provide a single starting point to access internal and external resources. In its simplest form, an intranet is established with the technologies for local area networks (LANs) and wide area networks (WANs). Many modern intranets have search engines, user profiles, blogs, mobile apps with notifications, and events planning within their infrastructure. An intranet is sometimes contrasted to an extranet. While an intranet is generally restricted to employees of the organization, extranets may also be accessed by customers, suppliers, or other approved parties. Extranets extend a private network onto the Internet with special provisions for authentication, authorization and accounting (AAA protocol). == Uses == Intranets are increasingly being used to deliver tools, such as for collaboration (to facilitate working in groups and teleconferencing) or corporate directories, sales and customer relationship management, or project management. Intranets are also used as corporate culture-change platforms. For example, a large number of employees using an intranet forum application to host a discussion about key issues could come up with new ideas related to management, productivity, quality, and other corporate issues. In large intranets, website traffic is often similar to public website traffic and can be better understood by using web metrics software to track overall activity. User surveys also improve intranet website effectiveness. Larger businesses allow users within their intranet to access public internet through firewall servers. They have the ability to screen incoming and outgoing messages, keeping security intact. When part of an intranet is made accessible to customers and others outside the business, it becomes part of an extranet. Businesses can send private messages through the public network using special encryption/decryption and other security safeguards to connect one part of their intranet to another. Intranet user-experience, editorial, and technology teams work together to produce in-house sites. Most commonly, intranets are managed by the communications, HR or CIO departments of large organizations, or some combination of these. Because of the scope and variety of content and the number of system interfaces, the intranets of many organizations are much more complex than their respective public websites. Intranets and the use of intranets are growing rapidly. According to the Intranet Design Annual 2007 from Nielsen Norman Group, the number of pages on participants' intranets averaged 200,000 over the years 2001 to 2003 and has grown to an average of 6 million pages over 2005–2007. == Benefits == Intranets can help users locate and view information faster and use applications relevant to their roles and responsibilities. With a web browser interface, users can access data held in any database the organization wants to make available at any time and — subject to security provisions — from anywhere within company workstations, increasing employees' ability to perform their jobs faster, more accurately, and with confidence that they have the right information. It also helps improve services provided to users. Using hypermedia and Web technology, Web publishing allows for the maintenance of and easy access to cumbersome corporate knowledge, such as employee manuals, benefits documents, company policies, business standards, news feeds, and even training, all of which can be accessed throughout a company using common Internet standards (Acrobat files, Flash files, CGI applications). Because each business unit can update the online copy of a document, the most recent version is usually available to employees using the intranet. Intranets are also used as a platform for developing and deploying applications to support business operations and decisions across the internetworked enterprise. Information is easily accessible to all authorised users, enabling collaboration. Being able to communicate in real-time through integrated third-party tools, such as an instant messenger, promotes the sharing of ideas and removes blockages to communication to help boost a business's productivity. Intranets can serve as powerful tools for communicating (such as through chat, email and/or blogs) within a given organization about vertically strategic initiatives that have a global reach throughout said organization. The type of information that can easily be conveyed is the purpose of the initiative and what it is aiming to achieve, who is driving it, results achieved to date, and whom to speak to for more information. By providing this information on the intranet, staff can keep up-to-date with the strategic focus of their organization. For example, when Nestlé had a number of food processing plants in Scandinavia, their central support system had to deal with a number of queries every day. When Nestlé decided to invest in an intranet, they quickly realized the savings. Gerry McGovern says that the savings from the reduction in query calls was substantially greater than the investment in the intranet. Users can view information and data via a web browser rather than maintaining physical documents such as procedure manuals, internal phone list and requisition forms. This can potentially save the business money on printing, duplicating documents, and the environment, as well as document maintenance overhead. For example, the HRM company PeopleSoft "derived significant cost savings by shifting HR processes to the intranet". McGovern goes on to say the manual cost of enrolling in benefits was found to be US$109.48 per enrollment. "Shifting this process to the intranet reduced the cost per enrollment to $21.79; a saving of 80 percent". Another company that saved money on expense reports was Cisco. "In 1996, Cisco processed 54,000 reports and the amount of dollars processed was USD19 million". Many companies dictate computer specifications which, in turn, may allow Intranet developers to write applications that only have to work on one browser such that there are no cross-browser compatibility issues. Being able to specifically address one's "viewer" is a great advantage. Since intranets are user-specific (requiring database/network authentication prior to access), users know exactly who they are interfacing with and can personalize their intranet based on role (job title, department) or individual ("Congratulations Jane, on your 3rd year with our company!"). Since "involvement in decision making" is one of the main drivers of employee engagement, offering tools (like forums or surveys) that foster peer-to-peer collaboration and employee participation can make employees feel more valued and involved. == Planning and creation == Most organizations devote considerable resources into the planning and implementation of their intranet as it is of strategic importance to the organization's success. Some of the planning would include topics such as determining the purpose and goals of the intranet, identifying persons or departments responsible for implementation and management and devising functional plans, page layouts and designs. The appropriate staff would also ensure that implementation schedules and phase-out of existing systems were organized, while defining and implementing security of the intranet and ensuring it lies within legal boundaries and other constraints. In order to produce a high-value end product, systems planners should determine the level of interactivity (e.g. wikis, on-line forms) desired. Planners may also consider whether the input of new data and updating of existing data is to be centrally controlled or devolve. These decisions sit alongside to the hardware and software considerations (like content management systems), participation issues (like good taste, harassment, confidentiality), and features to be supported. Intranets are often static sites; they are a shared drive, serving up centrally stored documents alongside internal articles or communications (often one-way communication). By leveraging firms which specialise in 'social' intranets, organisations are beginning to think of how their intranets can become a 'communication hub' for their entire team. The actual implementation would include steps such as securing senior management support and funding, conducting a business requirement analysis and identifying users' information needs. From the technical perspective, there would need to be a coordinated installation of the web server and user access netw

    Read more →
  • Application delivery network

    Application delivery network

    An application delivery network (ADN) is a suite of technologies that, when deployed together, provide availability, security, visibility, and acceleration for Internet applications such as websites. ADN components provide supporting functionality that enables website content to be delivered to visitors and other users of that website, in a fast, secure, and reliable way. Gartner defines application delivery networking as the combination of WAN optimization controllers (WOCs) and application delivery controllers (ADCs). At the data center end of an ADN is the ADC, an advanced traffic management device that is often also referred to as a web switch, content switch, or multilayer switch, the purpose of which is to distribute traffic among a number of servers or geographically dislocated sites based on application specific criteria. In the branch office portion of an ADN is the WAN optimization controller, which works to reduce the number of bits that flow over the network using caching and compression, and shapes TCP traffic using prioritization and other optimization techniques. Some WOC components are installed on PCs or mobile clients, and there is typically a portion of the WOC installed in the data center. Application delivery networks are also offered by some CDN vendors. The ADC, one component of an ADN, evolved from layer 4-7 switches in the late 1990s when it became apparent that traditional load balancing techniques were not robust enough to handle the increasingly complex mix of application traffic being delivered over a wider variety of network connectivity options. == Application delivery techniques == The Internet was designed according to the end-to-end principle. This principle keeps the core network relatively simple and moves the intelligence as much as possible to the network end-points: the hosts and clients. An Application Delivery Network (ADN) enhances the delivery of applications across the Internet by employing a number of optimization techniques. Many of these techniques are based on established best-practices employed to efficiently route traffic at the network layer including redundancy and load balancing In theory, an Application Delivery Network (ADN) is closely related to a content delivery network. The difference between the two delivery networks lies in the intelligence of the ADN to understand and optimize applications, usually referred to as application fluency. Application Fluent Network (AFN) is based on the concept of Application Fluency to refer to WAN optimization techniques applied at Layer Four to Layer Seven of the OSI model for networks. Application Fluency implies that the network is fluent or intelligent in understanding and being able to optimize delivery of each application. Application Fluent Network is an addition of SDN capabilities. The acronym 'AFN' is used by Alcatel-Lucent Enterprise to refer to an Application Fluent Network. Application delivery uses one or more layer 4–7 switches, also known as a web switch, content switch, or multilayer switch to intelligently distribute traffic to a pool, also known as a cluster or farm, of servers. The application delivery controller (ADC) is assigned a single virtual IP address (VIP) that represents the pool of servers. Traffic arriving at the ADC is then directed to one of the servers in the pool (cluster, farm) based on a number of factors including application specific data values, application transport protocol, availability of servers, current performance metrics, and client-specific parameters. An ADN provides the advantages of load distribution, increase in capacity of servers, improved scalability, security, and increased reliability through application specific health checks. Increasingly the ADN comprises a redundant pair of ADC on which is integrated a number of different feature sets designed to provide security, availability, reliability, and acceleration functions. In some cases these devices are still separate entities, deployed together as a network of devices through which application traffic is delivered, each providing specific functionality that enhances the delivery of the application. == ADN optimization techniques == === TCP multiplexing === TCP Multiplexing is loosely based on established connection pooling techniques utilized by application server platforms to optimize the execution of database queries from within applications. An ADC establishes a number of connections to the servers in its pool and keeps the connections open. When a request is received by the ADC from the client, the request is evaluated and then directed to a server over an existing connection. This has the effect of reducing the overhead imposed by establishing and tearing down the TCP connection with the server, improving the responsiveness of the application. Some ADN implementations take this technique one step further and also multiplex HTTP and application requests. This has the benefit of executing requests in parallel, which enhances the performance of the application. === TCP optimization === There are a number of Request for Comments (RFCs) which describe mechanisms for improving the performance of TCP. Many ADN implement these RFCs in order to provide enhanced delivery of applications through more efficient use of TCP. The RFCs most commonly implemented are: Delayed Acknowledgements Nagle Algorithm Selective Acknowledgements Explicit Congestion Notification ECN Limited and Fast Retransmits Adaptive Initial Congestion Windows === Data compression and caching === ADNs also provide optimization of application data through caching and compression techniques. There are two types of compression used by ADNs today: industry standard HTTP compression and proprietary data reduction algorithms. It is important to note that the cost in CPU cycles to compress data when traversing a LAN can result in a negative performance impact and therefore best practices are to only utilize compression when delivering applications via a WAN or particularly congested high-speed data link. HTTP compression is asymmetric and transparent to the client. Support for HTTP compression is built into web servers and web browsers. All commercial ADN products currently support HTTP compression. A second compression technique is achieved through data reduction algorithms. Because these algorithms are proprietary and modify the application traffic, they are symmetric and require a device to reassemble the application traffic before the client can receive it. A separate class of devices known as WAN Optimization Controllers (WOC) provide this functionality, but the technology has been slowly added to the ADN portfolio over the past few years as this class of device continues to become more application aware, providing additional features for specific applications such as CIFS and SMB. == ADN reliability and availability techniques == === Advanced health checking === Advanced health checking is the ability of an ADN to determine not only the state of the server on which an application is hosted, but the status of the application it is delivering. Advanced health checking techniques allow the ADC to intelligently determine whether or not the content being returned by the server is correct and should be delivered to the client. This feature enables other reliability features in the ADN, such as resending a request to a different server if the content returned by the original server is found to be erroneous. === Load balancing algorithms === The load balancing algorithms found in today's ADN are far more advanced than the simplistic round-robin and least connections algorithms used in the early 1990s. These algorithms were originally loosely based on operating systems' scheduling algorithms, but have since evolved to factor in conditions peculiar to networking and application environments. It is more accurate to describe today's "load balancing" algorithms as application routing algorithms, as most ADN employ application awareness to determine whether an application is available to respond to a request. This includes the ability of the ADN to determine not only whether the application is available, but whether or not the application can respond to the request within specified parameters, often referred to as a service level agreement. Typical industry standard load balancing algorithms available today include: Round Robin Least Connections Fastest Response Time Weighted Round Robin Weighted Least Connections Custom values assigned to individual servers in a pool based on SNMP or other communication mechanism === Fault tolerance === The ADN provides fault tolerance at the server level, within pools or farms. This is accomplished by designating specific servers as a 'backup' that is activated automatically by the ADN in the event that the primary server(s) in the pool fail. The ADN also ensures application availability and reliability through its ability to seamlessly "failover"

    Read more →
  • Control break

    Control break

    In computer programming, a control break is a change in the value of one of the keys on which a file is sorted, which requires some extra processing. For example, with an input file sorted by post code, the number of items found in each postal district might need to be printed on a report, and a heading shown for the next district. Quite often there is a hierarchy of nested control breaks in a program, such as streets within districts within areas, with the need for a grand total at the end. Structured programming techniques have been developed to ensure correct processing of control breaks in languages such as COBOL and to ensure that conditions such as empty input files and sequence errors are handled properly. With fourth-generation languages such as SQL, the programming language should handle most of the details of control breaks automatically.

    Read more →
  • Retrieval-augmented generation

    Retrieval-augmented generation

    Retrieval-augmented generation (RAG) is a technique that enables large language models (LLMs) to retrieve and incorporate new information from external data sources. With RAG, LLMs first refer to a specified set of documents, then respond to user queries. These documents supplement information from the LLM's pre-existing training data. This allows LLMs to use domain-specific and/or updated information that is not available in the training data. For example, this enables LLM-based chatbots to access internal company data or generate responses based on authoritative sources. RAG improves LLMs by incorporating information retrieval before generating responses. Unlike LLMs that rely on static training data, RAG pulls relevant text from databases, uploaded documents, or web sources. According to Ars Technica, "RAG is a way of improving LLM performance, in essence by blending the LLM process with a web search or other document look-up process to help LLMs stick to the facts." This method helps reduce AI hallucinations, which have caused chatbots to describe policies that don't exist, or recommend nonexistent legal cases to lawyers that are looking for citations to support their arguments. RAG also reduces the need to retrain LLMs with new data, saving on computational and financial costs. Beyond efficiency gains, RAG also allows LLMs to include sources in their responses, so users can verify the cited sources. This provides greater transparency, as users can cross-check retrieved content to ensure accuracy and relevance. The term retrieval-augmented generation (RAG) was introduced in a 2020 paper that described combining a parametric language model with a non-parametric external memory accessed through retrieval at inference time. == RAG and LLM limitations == LLMs can provide incorrect information. For example, when Google first demonstrated its LLM tool "Google Bard" (later re-branded to Gemini), the LLM provided incorrect information about the James Webb Space Telescope. This error contributed to a $100 billion decline in Google's stock value. RAG is used to prevent these errors, but it does not solve all the problems. For example, LLMs can generate misinformation even when pulling from factually correct sources if they misinterpret the context. MIT Technology Review gives the example of an AI-generated response stating, "The United States has had one Muslim president, Barack Hussein Obama." The model retrieved this from an academic book rhetorically titled Barack Hussein Obama: America's First Muslim President? The LLM did not "know" or "understand" the context of the title, generating a false statement. LLMs with RAG are programmed to prioritize new information. This technique has been called "prompt stuffing." Without prompt stuffing, the LLM's input is generated by a user; with prompt stuffing, additional relevant context is added to this input to guide the model's response. This approach provides the LLM with key information early in the prompt, encouraging it to prioritize the supplied data over pre-existing training knowledge. == Process == Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating an information-retrieval mechanism that allows models to access and utilize additional data beyond their original training set. Ars Technica notes that "when new information becomes available, rather than having to retrain the model, all that's needed is to augment the model's external knowledge base with the updated information" ("augmentation"). IBM states that "in the generative phase, the LLM draws from the augmented prompt and its internal representation of its training data to synthesize" an answer. === RAG key stages === Typically, the data to be referenced is converted into LLM embeddings, numerical representations in the form of a large vector space. RAG can be used on unstructured (usually text), semi-structured, or structured data (for example knowledge graphs). These embeddings are then stored in a vector database to allow for document retrieval. Given a user query, a document retriever is first called to select the most relevant documents that will be used to augment the query. This comparison can be done using a variety of methods, which depend in part on the type of indexing used. The model feeds this relevant retrieved information into the LLM via prompt engineering of the user's original query. Newer implementations (as of 2023) can also incorporate specific augmentation modules with abilities such as expanding queries into multiple domains and using memory and self-improvement to learn from previous retrievals. Finally, the LLM can generate output based on both the query and the retrieved documents. Some models incorporate extra steps to improve output, such as the re-ranking of retrieved information, context selection, and fine-tuning. == Applications == Retrieval-augmented generation is used in applications where generated responses need to be grounded in external or frequently updated information. Commonly cited use cases include search engines, question-answering systems, customer support chatbots, enterprise knowledge assistants, content generation, recommendation systems, retail and e-commerce, and industrial or manufacturing workflows. In healthcare, RAG has been studied as a way to ground large language model outputs in external medical knowledge sources, although reviews have noted continuing challenges around evaluation, ethics, and clinical reliability. == Improvements == Improvements to the basic process above can be applied at different stages in the RAG flow. === Encoder === These methods focus on the encoding of text as either dense or sparse vectors. Sparse vectors, which encode the identity of a word, are typically dictionary-length and contain mostly zeros. Dense vectors, which encode meaning, are more compact and contain fewer zeros. Various enhancements can improve the way similarities are calculated in the vector stores (databases). Performance improves by optimizing how vector similarities are calculated. Dot products enhance similarity scoring, while approximate nearest neighbor (ANN) searches improve retrieval efficiency over K-nearest neighbors (KNN) searches. Accuracy may be improved with Late Interactions, which allow the system to compare words more precisely after retrieval. This helps refine document ranking and improve search relevance. Hybrid vector approaches may be used to combine dense vector representations with sparse one-hot vectors, taking advantage of the computational efficiency of sparse dot products over dense vector operations. Other retrieval techniques focus on improving accuracy by refining how documents are selected. Some retrieval methods combine sparse representations, such as SPLADE, with query expansion strategies to improve search accuracy and recall. === Retriever-centric methods === These methods aim to enhance the quality of document retrieval in vector databases: Pre-training the retriever using the Inverse Cloze Task (ICT), a technique that helps the model learn retrieval patterns by predicting masked text within documents. Supervised retriever optimization aligns retrieval probabilities with the generator model's likelihood distribution. This involves retrieving the top-k vectors for a given prompt, scoring the generated response's perplexity, and minimizing KL divergence between the retriever's selections and the model's likelihoods to refine retrieval. Reranking techniques can refine retriever performance by prioritizing the most relevant retrieved documents during training. === Language model === By redesigning the language model with the retriever in mind, a 25-time smaller network can get comparable perplexity as its much larger counterparts. Because it is trained from scratch, this method (Retro) incurs the high cost of training runs that the original RAG scheme avoided. The hypothesis is that by giving domain knowledge during training, Retro needs less focus on the domain and can devote its smaller weight resources only to language semantics. The redesigned language model is shown here. It has been reported that Retro is not reproducible, so modifications were made to make it so. The more reproducible version is called Retro++ and includes in-context RAG. === Chunking === Chunking involves various strategies for breaking up the data into vectors so the retriever can find details in it. Three types of chunking strategies are: Fixed length with overlap. This is fast and easy. Overlapping consecutive chunks helps to maintain semantic context across chunks. Syntax-based chunks can break the document up into sentences. Libraries such as spaCy or NLTK can also help. File format-based chunking. Certain file types have natural chunks built in, and it's best to respect them. For example, code files are best chunked and vectorized as whole functions or classes. HTML files should leave

    or base64 encoded elements

    Read more →
  • Undeniable signature

    Undeniable signature

    An undeniable signature is a digital signature scheme which allows the signer to be selective to whom they allow to verify signatures. The scheme adds explicit signature repudiation, preventing a signer later refusing to verify a signature by omission; a situation that would devalue the signature in the eyes of the verifier. It was invented by David Chaum and Hans van Antwerpen in 1989. == Overview == In this scheme, a signer possessing a private key can publish a signature of a message. However, the signature reveals nothing to a recipient/verifier of the message and signature without taking part in either of two interactive protocols: Confirmation protocol, which confirms that a candidate is a valid signature of the message issued by the signer, identified by the public key. Disavowal protocol, which confirms that a candidate is not a valid signature of the message issued by the signer. The motivation for the scheme is to allow the signer to choose to whom signatures are verified. However, that the signer might claim the signature is invalid at any later point, by refusing to take part in verification, would devalue signatures to verifiers. The disavowal protocol distinguishes these cases removing the signer's plausible deniability. It is important that the confirmation and disavowal exchanges are not transferable. They achieve this by having the property of zero-knowledge; both parties can create transcripts of both confirmation and disavowal that are indistinguishable, to a third-party, of correct exchanges. The designated verifier signature scheme improves upon deniable signatures by allowing, for each signature, the interactive portion of the scheme to be offloaded onto another party, a designated verifier, reducing the burden on the signer. == Zero-knowledge protocol == The following protocol was suggested by David Chaum. A group, G, is chosen in which the discrete logarithm problem is intractable, and all operation in the scheme take place in this group. Commonly, this will be the finite cyclic group of order p contained in Z/nZ, with p being a large prime number; this group is equipped with the group operation of integer multiplication modulo n. An arbitrary primitive element (or generator), g, of G is chosen; computed powers of g then combine obeying fixed axioms. Alice generates a key pair, randomly chooses a private key, x, and then derives and publishes the public key, y = gx. === Message signing === Alice signs the message, m, by computing and publishing the signature, z = mx. === Confirmation (i.e., avowal) protocol === Bob wishes to verify the signature, z, of m by Alice under the key, y. Bob picks two random numbers: a and b, and uses them to blind the message, sending to Alice: c = magb. Alice picks a random number, q, uses it to blind, c, and then signing this using her private key, x, sending to Bob: s1 = cgq ands2 = s1x. Note that s1x = (cgq)x = (magb)xgqx = (mx)a(gx)b+q = zayb+q. Bob reveals a and b. Alice verifies that a and b are the correct blind values, then, if so, reveals q. Revealing these blinds makes the exchange zero knowledge. Bob verifies s1 = cgq, proving q has not been chosen dishonestly, and s2 = zayb+q, proving z is valid signature issued by Alice's key. Note that zayb+q = (mx)a(gx)b+q. Alice can cheat at step 2 by attempting to randomly guess s2. === Disavowal protocol === Alice wishes to convince Bob that z is not a valid signature of m under the key, gx; i.e., z ≠ mx. Alice and Bob have agreed an integer, k, which sets the computational burden on Alice and the likelihood that she should succeed by chance. Bob picks random values, s ∈ {0, 1, ..., k} and a, and sends: v1 = msga and v2 = zsya, where exponentiating by a is used to blind the sent values. Note that v2 = zsya = (mx)s(gx)a = v1x. Alice, using her private key, computes v1x and then the quotient, v1xv2−1 = (msga)x(zsgxa)−1 = msxz−s = (mxz−1)s. Thus, v1xv2−1 = 1, unless z ≠ mx. Alice then tests v1xv2−1 for equality against the values: (mxz−1)i for i ∈ {0, 1, …, k}; which are calculated by repeated multiplication of mxz−1 (rather than exponentiating for each i). If the test succeeds, Alice conjectures the relevant i to be s; otherwise, she conjectures random value. Where z = mx, (mxz−1)i = v1xv2−1 = 1 for all i, s is unrecoverable. Alice commits to i: she picks a random r and sends hash(r, i) to Bob. Bob reveals a. Alice confirms that a is the correct blind (i.e., v1 and v2 can be generated using it), then, if so, reveals r. Revealing these blinds makes the exchange zero knowledge. Bob checks hash(r, i) = hash(r, s), proving Alice knows s, hence z ≠ mx. If Alice attempts to cheat at step 3 by guessing s at random, the probability of succeeding is 1/(k + 1). So, if k = 1023 and the protocol is conducted ten times, her chances are 1 to 2100.

    Read more →
  • Data refuge

    Data refuge

    Data Refuge is a public and collaborative project designed to address concerns about federal climate and environmental data that is in danger of being lost. In particular, the initiative addresses five main concerns: What are the best ways to safeguard data? How do federal agencies play a crucial role in collecting, managing, and distributing data? How do government priorities impact data's accessibility? Which projects and research fields depend on federal data? Which data sets are of value to research and local communities, and why? Data Refuge began as a grassroots organization in opposition to government data on climate change and the environment not being archived systemically. Data Refuge's main goal is to collect and allocate data in multiple safe locations to create a sustainable way of archiving old and new data. Data Refuge was initiated in 2016 to protect federal climate and environmental data that is vulnerable under an administration that denies climate change. The system aims to make public research-quality copies of federal climate and environmental data. Data Refuge is supported by the National Geographic Foundation, private donors, Libraries+ Network, Preserving Electronic Governance Initiative (PEGI), the Union of Concerned Scientists (USC), and the Penn Program in Environmental Humanities (PPEH). == Types of data == Data Refuge collects public federal data on the climate and environment in the form of satellite imagery, PDFs, and stories. The data are stored in multiple trusted locations as they are less vulnerable if in only one location, and to ensure accessibility for researchers. Through the Data Rescue events, Data Refuge has accumulated 4 terabytes of data, 30,000 URLs, and 800 participants. === Storytelling === Data Refuge collects stories on vulnerable federal climate and environmental data through: surveys, oral history, photo essays, maps, video shorts, and animations. The stories are archived in a public bank that showcase how federal environmental data support health and safety in communities. Data Stories are collected at Data Rescue events, which are partnered with universities, city and town halls, and advocacy groups. Data stories are collected and used to emphasize the importance of Data Refuge, in how the data on climate change and the environment are being used by people in the United States and across the world for meaningful practices.

    Read more →