Apache Pig is a high-level platform for creating programs that run on Apache Hadoop. The language for this platform is called Pig Latin. Pig can execute its Hadoop jobs in MapReduce, Apache Tez, or Apache Spark. Pig Latin abstracts the programming from the Java MapReduce idiom into a notation which makes MapReduce programming high level, similar to that of SQL for relational database management systems. Pig Latin can be extended using user-defined functions (UDFs) which the user can write in Java, Python, JavaScript, Ruby or Groovy and then call directly from the language. == History == Apache Pig was originally developed at Yahoo Research around 2006 for researchers to have an ad hoc way of creating and executing MapReduce jobs on very large data sets. In 2007, it was moved into the Apache Software Foundation. === Naming === Regarding the naming of the Pig programming language, the name was chosen arbitrarily and stuck because it was memorable, easy to spell, and for novelty. The story goes that the researchers working on the project initially referred to it simply as 'the language'. Eventually they needed to call it something. Off the top of his head, one researcher suggested Pig, and the name stuck. It is quirky yet memorable and easy to spell. While some have hinted that the name sounds coy or silly, it has provided us with an entertaining nomenclature, such as Pig Latin for the language, Grunt for the shell, and PiggyBank for the CPAN-like shared repository. == Example == Below is an example of a "Word Count" program in Pig Latin: The above program will generate parallel executable tasks which can be distributed across multiple machines in a Hadoop cluster to count the number of words in a dataset such as all the webpages on the internet. == Pig vs SQL == In comparison to SQL, Pig has a nested relational model, uses lazy evaluation, uses extract, transform, load (ETL), is able to store data at any point during a pipeline, declares execution plans, supports pipeline splits, thus allowing workflows to proceed along DAGs instead of strictly sequential pipelines. On the other hand, it has been argued DBMSs are substantially faster than the MapReduce system once the data is loaded, but that loading the data takes considerably longer in the database systems. It has also been argued RDBMSs offer out of the box support for column-storage, working with compressed data, indexes for efficient random data access, and transaction-level fault tolerance. Pig Latin is procedural and fits very naturally in the pipeline paradigm while SQL is instead declarative. In SQL users can specify that data from two tables must be joined, but not what join implementation to use (You can specify the implementation of JOIN in SQL, thus "... for many SQL applications the query writer may not have enough knowledge of the data or enough expertise to specify an appropriate join algorithm."). Pig Latin allows users to specify an implementation or aspects of an implementation to be used in executing a script in several ways. In effect, Pig Latin programming is similar to specifying a query execution plan, making it easier for programmers to explicitly control the flow of their data processing task. SQL is oriented around queries that produce a single result. SQL handles trees naturally, but has no built in mechanism for splitting a data processing stream and applying different operators to each sub-stream. Pig Latin script describes a directed acyclic graph (DAG) rather than a pipeline. Pig Latin's ability to include user code at any point in the pipeline is useful for pipeline development. If SQL is used, data must first be imported into the database, and then the cleansing and transformation process can begin.
Ciscogate
Ciscogate, also known as the Black Hat Bug, is the name given to a legal incident that occurred at the Black Hat Briefings security conference in Las Vegas, Nevada, on July 27, 2005. On the morning of the first day of the conference, July 26, 2005, some attendees noticed that 30 pages of text had been physically ripped out of the extensive conference presentation booklet the night before at the request of Cisco Systems and the CD-ROM with presentation slides was not included. It was determined the pages covered a talk to be given by Michael Lynn, a security researcher with Atlanta-based IBM Internet Security Systems (ISS). Instead of the pages with the details, attendees found a photographed copy of a notice from Black Hat saying "Due to some last minute changes beyond Black Hat's control, and at the request of the presenter, the included materials aren't up to the standards Black Hat tries to meet. Black Hat will be the first to apologize. We hope the vendors involved will follow suit." According to Lynn's lawyer, his employer had approved of the talk leading up to the conference but changed their minds two days before the scheduled talk, forbidding him from presenting. Lynn's original presentation was to cover a vulnerability in Cisco routers. The presentation was one of four scheduled to follow Jeff Moss' keynote address on the first day of the conference, titled "Cisco IOS Security Architecture". After being told by his employer that he could not present on the topic, Lynn chose an alternate topic. Cisco and ISS had offered to give new joint presentation but this was turned down by Black Hat because the original speaking slot was given to Lynn, not Cisco. Lynn's presentation began by covering security issues in services that allow users to make Voice over IP telephone calls. Shortly after beginning the presentation Lynn changed back to his original topic and began disclosing some technical details of the vulnerability he found in Cisco routers stating that he would rather resign from his job at ISS than keep the details private. == Lawsuit == Shortly after Lynn concluded his talk he met Jennifer Granick, who would soon become his lawyer. During their initial meeting Lynn told Granick that he expected to be sued. Later in the evening Lynn had heard that Cisco and ISS had filed a lawsuit and requested a temporary restraining order against Black Hat but not himself. A public relations representative from Black Hat told Granick that the lawsuit was against both Black Hat and Lynn and that the companies had scheduled an Ex parte hearing in San Francisco the next morning to request the restraining order. That night, Andrew Valentine, an attorney for ISS and Cisco called Lynn who directed them to Granick. During the conversation Valentine explained the claims and accusations against Lynn, which included three things: 1) ISS claimed copyright over the presentation that Lynn gave, 2) Cisco claimed copyright over the decompiled machine code obtained from the router which was included in the presentation, and 3) Cisco claimed the presentation contained trade secrets. These complaints were outlined in a civil complaint at the U.S. Northern District of California and filed against both Lynn and Black Hat. According to Granick, she and Valentine were able agree to an injunction to settle the case without court proceedings. This deal was almost called off due to an inadvertent mistake by Black Hat in which they had restored Lynn's presentation on their web server. Black Hat, Granick, and the plaintiff's lawyers were able to resolve this problem and the deal stood. One condition of the settlement required Lynn to provide an image of all computer data he used in his research to be provided to a third party for forensic analysis before erasing his research and any Cisco data from his systems. The settlement also stipulated that Lynn was prohibited from talking about the vulnerability in the future. == FBI Investigation == Shortly after lawyers for Lynn and ISS / Cisco filed settlement papers, FBI agents from the Las Vegas office arrived at the conference to begin asking questions. According to Granick, they were there at the request of the Atlanta FBI office and Lynn was not of interest. Granick asserted the Fifth and Sixth amendment rights on behalf of her client, Lynn. Granick asserted his rights for the Atlanta office and asked if an arrest warrant had been issued for Lynn. Over the next 24 hours Granick was not able to ascertain the status of a warrant but ultimately determined no warrant was issued. When the FBI was asked about the case by a journalist, spokesman Paul Bresson declined to discuss the case saying "Our policy is to not make any comment on anything that is ongoing. That's not to confirm that something is, because I really don't know". Granick would only confirm to journalists that the "investigation has to do with the presentation". == Response == === Attendees === Attendees of Black Hat Briefings, as well as many that also attended DEF CON, were not happy with vendors threatening legal action over vulnerability disclosure. The term "Ciscogate" was coined quickly by an unknown person, but some attendees were quick to create shirts to commemorate the incident. === Cisco === Mojgan Khalili, a senior manager for corporate PR at Cisco, issued a statement to the press saying "It is important to note that the information Mr. Lynn presented was not a disclosure of a new vulnerability or a flaw with Cisco IOS software. Mr. Lynn's research explores possible ways to expand exploitations of existing security vulnerabilities impacting routers." === ISS === Kim Duffy, managing director of ISS Australia, was asked about ISS's response to the incident. Duffy responded that it was "business as usual" as the company handled the incident "strictly by the book". He gave a brief statement to ZDNet UK saying "ISS has published rules for disclosure and that is what we stick to. We didn't care to publish [the disclosure] because we were not ready. We had not completed the research to our satisfaction so it was not ready to be disclosed". ISS spokesperson Roger Fortier confirmed that Lynn was no longer employed with the company and that ISS was still working with Cisco on the matter. He gave a statement to the Washington Post saying "ISS and Cisco have been working on this in the background and didn't feel at this time that the material was ready for publication. The decision was made on Monday to pull the presentation because we wanted to make sure the research was fully baked."
Voice activity detection
Voice activity detection (VAD), also known as speech activity detection or speech detection, is the detection of the presence or absence of human speech, used in speech processing. The main uses of VAD are in speaker diarization, speech coding and speech recognition. It can facilitate speech processing, and can also be used to deactivate some processes during non-speech section of an audio session: it can avoid unnecessary coding/transmission of silence packets in Voice over Internet Protocol (VoIP) applications, saving on computation and on network bandwidth. VAD is an important enabling technology for a variety of speech-based applications. Therefore, various VAD algorithms have been developed that provide varying features and compromises between latency, sensitivity, accuracy and computational cost. Some VAD algorithms also provide further analysis, for example whether the speech is voiced, unvoiced or sustained. Voice activity detection is usually independent of language. It was first investigated for use on time-assignment speech interpolation (TASI) systems. == Algorithm overview == The typical design of a VAD algorithm is as follows: There may first be a noise reduction stage, e.g. via spectral subtraction. Then some features or quantities are calculated from a section of the input signal. A classification rule is applied to classify the section as speech or non-speech – often this classification rule finds when a value exceeds a certain threshold. There may be some feedback in this sequence, in which the VAD decision is used to improve the noise estimate in the noise reduction stage, or to adaptively vary the threshold(s). These feedback operations improve the VAD performance in non-stationary noise (i.e. when the noise varies a lot). A representative set of recently published VAD methods formulates the decision rule on a frame by frame basis using instantaneous measures of the divergence distance between speech and noise. The different measures which are used in VAD methods include spectral slope, correlation coefficients, log likelihood ratio, cepstral, weighted cepstral, and modified distance measures. Independently from the choice of VAD algorithm, a compromise must be made between having voice detected as noise, or noise detected as voice (between false positive and false negative). A VAD operating in a mobile phone must be able to detect speech in the presence of a range of very diverse types of acoustic background noise. In these difficult detection conditions it is often preferable that a VAD should fail-safe, indicating speech detected when the decision is in doubt, to lower the chance of losing speech segments. The biggest difficulty in the detection of speech in this environment is the very low signal-to-noise ratios (SNRs) that are encountered. It may be impossible to distinguish between speech and noise using simple level detection techniques when parts of the speech utterance are buried below the noise. == Applications == VAD is an integral part of different speech communication systems such as audio conferencing, echo cancellation, speech recognition, speech encoding, speaker recognition and hands-free telephony. In the field of multimedia applications, VAD allows simultaneous voice and data applications. Similarly, in Universal Mobile Telecommunications Systems (UMTS), it controls and reduces the average bit rate and enhances overall coding quality of speech. In cellular radio systems (for instance GSM and CDMA systems) based on Discontinuous Transmission (DTX) mode, VAD is essential for enhancing system capacity by reducing co-channel interference and power consumption in portable digital devices. In speech processing applications, voice activity detection plays an important role since non-speech frames are often discarded. For a wide range of applications such as digital mobile radio, Digital Simultaneous Voice and Data (DSVD) or speech storage, it is desirable to provide a discontinuous transmission of speech-coding parameters. Advantages can include lower average power consumption in mobile handsets, higher average bit rate for simultaneous services like data transmission, or a higher capacity on storage chips. However, the improvement depends mainly on the percentage of pauses during speech and the reliability of the VAD used to detect these intervals. On the one hand, it is advantageous to have a low percentage of speech activity. On the other hand, clipping, that is the loss of milliseconds of active speech, should be minimized to preserve quality. This is the crucial problem for a VAD algorithm under heavy noise conditions. === Use in telemarketing === One controversial application of VAD is in conjunction with predictive dialers used by telemarketing firms. In order to maximize agent productivity, telemarketing firms set up predictive dialers to call more numbers than they have agents available, knowing most calls will end up in either "Ring – No Answer" or answering machines. When a person answers, they typically speak briefly ("Hello", "Good evening", etc.) and then there is a brief period of silence. Answering machine messages are usually 3–15 seconds of continuous speech. By setting VAD parameters correctly, dialers can determine whether a person or a machine answered the call and, if it's a person, transfer the call to an available agent. If it detects an answering machine message, the dialer hangs up. Often, even when the system correctly detects a person answering the call, no agent may be available, resulting in a "silent call". Call screening with a multi-second message like "please say who you are, and I may pick up the phone" will frustrate such automated calls. == Performance evaluation == To evaluate a VAD, its output using test recordings is compared with those of an "ideal" VAD – created by hand-annotating the presence or absence of voice in the recordings. The performance of a VAD is commonly evaluated on the basis of the following four parameters: FEC (Front End Clipping): clipping introduced in passing from noise to speech activity; MSC (Mid Speech Clipping): clipping due to speech misclassified as noise; OVER: noise interpreted as speech due to the VAD flag remaining active in passing from speech activity to noise; NDS (Noise Detected as Speech): noise interpreted as speech within a silence period. Although the method described above provides useful objective information concerning the performance of a VAD, it is only an approximate measure of the subjective effect. For example, the effects of speech signal clipping can at times be hidden by the presence of background noise, depending on the model chosen for the comfort noise synthesis, so some of the clipping measured with objective tests is in reality not audible. It is therefore important to carry out subjective tests on VADs, the main aim of which is to ensure that the clipping perceived is acceptable. In VoIP applications, front-end clipping can be reduced by rewinding to shortly before the detection and sending very slightly delayed data. This kind of test requires a certain number of listeners to judge recordings containing the processing results of the VADs being tested, giving marks to several speech sequences on the following features: Quality; Comprehension difficulty; Audibility of clipping. These marks are then used to calculate average results for each of the features listed above, thus providing a global estimate of the behavior of the VAD being tested. To conclude, whereas objective methods are very useful in an initial stage to evaluate the quality of a VAD, subjective methods are more significant. As they require the participation of several people for a few days, increasing cost, they are generally only used when a proposal is about to be standardized. == Implementations == One early standard VAD is that developed by British Telecom for use in the Pan-European digital cellular mobile telephone service in 1991. It uses inverse filtering trained on non-speech segments to filter out background noise, so that it can then more reliably use a simple power-threshold to decide if a voice is present. The G.729 standard calculates the following features for its VAD: line spectral frequencies, full-band energy, low-band energy (<1 kHz), and zero-crossing rate. It applies a simple classification using a fixed decision boundary in the space defined by these features, and then applies smoothing and adaptive correction to improve the estimate. The GSM standard includes two VAD options developed by ETSI. Option 1 computes the SNR in nine bands and applies a threshold to these values. Option 2 calculates different parameters: channel power, voice metrics, and noise power. It then thresholds the voice metrics using a threshold that varies according to the estimated SNR. The Speex audio compression library uses a procedure named Improved Minima Controlled Recursive Averaging, which uses a smoothed representation of spectral pow
Structural similarity index measure
The structural similarity index measure (SSIM) is a method for predicting the perceived quality of digital television and cinematic pictures, as well as other kinds of digital images and videos. It is also used for measuring the similarity between two images. The SSIM index is a full reference metric; in other words, the measurement or prediction of image quality is based on an initial uncompressed or distortion-free image as reference. SSIM is a perception-based model that considers image degradation as perceived change in structural information, while also incorporating important perceptual phenomena, including both luminance masking and contrast masking terms. This distinguishes from other techniques such as mean squared error (MSE) or peak signal-to-noise ratio (PSNR) that instead estimate absolute errors. Structural information is the idea that the pixels have strong inter-dependencies especially when they are spatially close. These dependencies carry important information about the structure of the objects in the visual scene. Luminance masking is a phenomenon whereby image distortions (in this context) tend to be less visible in bright regions, while contrast masking is a phenomenon whereby distortions become less visible where there is significant activity or "texture" in the image. == History == The predecessor of SSIM was called Universal Quality Index (UQI), or Wang–Bovik index, which was developed by Zhou Wang and Alan Bovik in 2001. This evolved, through their collaboration with Hamid Sheikh and Eero Simoncelli, into the current version of SSIM, which was published in April 2004 in the IEEE Transactions on Image Processing. In addition to defining the SSIM quality index, the paper provides a general context for developing and evaluating perceptual quality measures, including connections to human visual neurobiology and perception, and direct validation of the index against human subject ratings. The basic model was developed in the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin and further developed jointly with the Laboratory for Computational Vision (LCV) at New York University. Further variants of the model have been developed in the Image and Visual Computing Laboratory at University of Waterloo and have been commercially marketed. SSIM subsequently found strong adoption in the image processing community and in the television and social media industries. The 2004 SSIM paper has been cited over 50,000 times according to Google Scholar, making it one of the highest cited papers in the image processing and video engineering fields. It was recognized with the IEEE Signal Processing Society Best Paper Award for 2009. It also received the IEEE Signal Processing Society Sustained Impact Award for 2016, indicative of a paper having an unusually high impact for at least 10 years following its publication. Because of its high adoption by the television industry, the authors of the original SSIM paper were each accorded a Primetime Engineering Emmy Award in 2015 by the Television Academy. == Algorithm == The SSIM index is calculated between two windows of pixel values x {\displaystyle x} and y {\displaystyle y} of common size, from corresponding locations in two images to be compared. These SSIM values can be aggregated across the full images by averaging or other variations. === Special-case formula === In one simple special case, further explained in the next section, the SSIM measure between x {\displaystyle x} and y {\displaystyle y} is: SSIM ( x , y ) = ( 2 μ x μ y + c 1 ) ( 2 σ x y + c 2 ) ( μ x 2 + μ y 2 + c 1 ) ( σ x 2 + σ y 2 + c 2 ) {\displaystyle {\hbox{SSIM}}(x,y)={\frac {(2\mu _{x}\mu _{y}+c_{1})(2\sigma _{xy}+c_{2})}{(\mu _{x}^{2}+\mu _{y}^{2}+c_{1})(\sigma _{x}^{2}+\sigma _{y}^{2}+c_{2})}}} with: μ x {\displaystyle \mu _{x}} the pixel sample mean of x {\displaystyle x} ; μ y {\displaystyle \mu _{y}} the pixel sample mean of y {\displaystyle y} ; σ x 2 {\displaystyle \sigma _{x}^{2}} the sample variance of x {\displaystyle x} ; σ y 2 {\displaystyle \sigma _{y}^{2}} the sample variance of y {\displaystyle y} ; σ x y {\displaystyle \sigma _{xy}} the sample covariance of x {\displaystyle x} and y {\displaystyle y} ; c 1 = ( k 1 L ) 2 {\displaystyle c_{1}=(k_{1}L)^{2}} , c 2 = ( k 2 L ) 2 {\displaystyle c_{2}=(k_{2}L)^{2}} two variables to stabilize the division with weak denominator; L {\displaystyle L} the dynamic range of the pixel-values (typically this is 2 # b i t s p e r p i x e l − 1 {\displaystyle 2^{\#bits\ per\ pixel}-1} ); k 1 = 0.01 {\displaystyle k_{1}=0.01} and k 2 = 0.03 {\displaystyle k_{2}=0.03} by default. === General formula and components === The SSIM formula is based on three comparison measurements between the samples of x {\displaystyle x} and y {\displaystyle y} : luminance ( l {\displaystyle l} ), contrast ( c {\displaystyle c} ), and structure ( s {\displaystyle s} ). The individual comparison functions are: l ( x , y ) = 2 μ x μ y + c 1 μ x 2 + μ y 2 + c 1 {\displaystyle l(x,y)={\frac {2\mu _{x}\mu _{y}+c_{1}}{\mu _{x}^{2}+\mu _{y}^{2}+c_{1}}}} c ( x , y ) = 2 σ x σ y + c 2 σ x 2 + σ y 2 + c 2 {\displaystyle c(x,y)={\frac {2\sigma _{x}\sigma _{y}+c_{2}}{\sigma _{x}^{2}+\sigma _{y}^{2}+c_{2}}}} s ( x , y ) = σ x y + c 3 σ x σ y + c 3 {\displaystyle s(x,y)={\frac {\sigma _{xy}+c_{3}}{\sigma _{x}\sigma _{y}+c_{3}}}} The SSIM for each block is then a weighted combination of those comparative measures: SSIM ( x , y ) = l ( x , y ) α ⋅ c ( x , y ) β ⋅ s ( x , y ) γ {\displaystyle {\text{SSIM}}(x,y)=l(x,y)^{\alpha }\cdot c(x,y)^{\beta }\cdot s(x,y)^{\gamma }} Choosing the third denominator stabilizing constant as: c 3 = c 2 / 2 {\displaystyle c_{3}=c_{2}/2} leads to a simplification when combining the c and s components with equal exponents ( β = γ {\displaystyle \beta =\gamma } ), as the numerator of c is then twice the denominator of s, leading to a cancellation leaving just a 2. Setting the weights (exponents) α , β , γ {\displaystyle \alpha ,\beta ,\gamma } to 1, the formula can then be reduced to the special case shown above. === Mathematical properties === SSIM satisfies the identity of indiscernibles, and symmetry properties, but not the triangle inequality or non-negativity, and thus is not a distance function. However, under certain conditions, SSIM may be converted to a normalized root MSE measure, which is a distance function. The square of such a function is not convex, but is locally convex and quasiconvex, making SSIM a feasible target for optimization. === Application of the formula === In order to evaluate the image quality, this formula is usually applied only on luma, although it may also be applied on color (e.g., RGB) values or chromatic (e.g. YCbCr) values. The resultant SSIM index is a decimal value between -1 and 1, where 1 indicates perfect similarity, 0 indicates no similarity, and -1 indicates perfect anti-correlation. For an image, it is typically calculated using a sliding Gaussian window of size 11×11 or a block window of size 8×8. The window can be displaced pixel-by-pixel on the image to create an SSIM quality map of the image. In the case of video quality assessment, the authors propose to use only a subgroup of the possible windows to reduce the complexity of the calculation. === Variants === ==== Multi-scale SSIM ==== A more advanced form of SSIM, called Multiscale SSIM (MS-SSIM) is conducted over multiple scales through a process of multiple stages of sub-sampling, reminiscent of multiscale processing in the early vision system. It has been shown to perform equally well or better than SSIM on different subjective image and video databases. ==== Multi-component SSIM ==== Three-component SSIM (3-SSIM) is a form of SSIM that takes into account the fact that the human eye can see differences more precisely on textured or edge regions than on smooth regions. The resulting metric is calculated as a weighted average of SSIM for three categories of regions: edges, textures, and smooth regions. The proposed weighting is 0.5 for edges, 0.25 for the textured and smooth regions. The authors mention that a 1/0/0 weighting (ignoring anything but edge distortions) leads to results that are closer to subjective ratings. This suggests that edge regions play a dominant role in image quality perception. The authors of 3-SSIM have also extended the model into four-component SSIM (4-SSIM). The edge types are further subdivided into preserved and changed edges by their distortion status. The proposed weighting is 0.25 for all four components. ==== Structural dissimilarity ==== Structural dissimilarity (DSSIM) may be derived from SSIM, though it does not constitute a distance function as the triangle inequality is not necessarily satisfied. DSSIM ( x , y ) = 1 − SSIM ( x , y ) 2 {\displaystyle {\hbox{DSSIM}}(x,y)={\frac {1-{\hbox{SSIM}}(x,y)}{2}}} ==== Video quality metrics and temporal variants ==== It is worth noting that the original vers
Mathematical morphology
Mathematical morphology (MM) is a theory and technique for analyzing and processing geometrical structures. It's based on set theory, lattice theory, topology, and random functions. MM is most commonly applied to digital images, but it can be employed as well on graphs, surface meshes, solids, and many other spatial structures. Topological and geometrical continuous-space concepts such as size, shape, convexity, connectivity, and geodesic distance, were introduced by MM on both continuous and discrete spaces. MM is also the foundation of morphological image processing, which consists of a set of operators that transform images according to the above characterizations. The basic morphological operators are erosion, dilation, opening and closing. MM was originally developed for binary images, and was later extended to grayscale functions and images. The subsequent generalization to complete lattices is widely accepted today as MM's theoretical foundation. == History == Mathematical Morphology was developed in 1964 by the collaborative work of Georges Matheron and Jean Serra, at the École des Mines de Paris, France. Matheron supervised the PhD thesis of Serra, devoted to the quantification of mineral characteristics from thin cross sections, and this work resulted in a novel practical approach, as well as theoretical advancements in integral geometry and topology. In 1968, the Centre de Morphologie Mathématique was founded by the École des Mines de Paris in Fontainebleau, France, led by Matheron and Serra. During the rest of the 1960s and most of the 1970s, MM dealt essentially with binary images, treated as sets, and generated a large number of binary operators and techniques: Hit-or-miss transform, dilation, erosion, opening, closing, granulometry, thinning, skeletonization, ultimate erosion, conditional bisector, and others. A random approach was also developed, based on novel image models. Most of the work in that period was developed in Fontainebleau. From the mid-1970s to mid-1980s, MM was generalized to grayscale functions and images as well. Besides extending the main concepts (such as dilation, erosion, etc.) to functions, this generalization yielded new operators, such as morphological gradients, top-hat transform and the Watershed (MM's main segmentation approach). In the 1980s and 1990s, MM gained a wider recognition, as research centers in several countries began to adopt and investigate the method. MM started to be applied to a large number of imaging problems and applications, especially in the field of non-linear filtering of noisy images. In 1986, Serra further generalized MM, this time to a theoretical framework based on complete lattices. This generalization brought flexibility to the theory, enabling its application to a much larger number of structures, including color images, video, graphs, meshes, etc. At the same time, Matheron and Serra also formulated a theory for morphological filtering, based on the new lattice framework. The 1990s and 2000s also saw further theoretical advancements, including the concepts of connections and levelings. In 1993, the first International Symposium on Mathematical Morphology (ISMM) took place in Barcelona, Spain. Since then, ISMMs are organized every 2–3 years: Fontainebleau, France (1994); Atlanta, USA (1996); Amsterdam, Netherlands (1998); Palo Alto, CA, USA (2000); Sydney, Australia (2002); Paris, France (2005); Rio de Janeiro, Brazil (2007); Groningen, Netherlands (2009); Intra (Verbania), Italy (2011); Uppsala, Sweden (2013); Reykjavík, Iceland (2015); Fontainebleau, France (2017); and Saarbrücken, Germany (2019). =
Amaq News Agency
Amaq News Agency (Arabic: وكالة أعماق الإخبارية, romanized: Wakālat Aʻmāq al-Ikhbārīyah) is a news outlet linked to the Islamic State (IS). Amaq is often the "first point of publication for claims of responsibility" for terrorist attacks in Western countries by the Islamic State. In March 2019, Amaq News Agency was designated as a foreign terrorist organization by the United States Department of State. == History == Among the founders of Amaq was Syrian journalist Baraa Kadek, who joined IS in late 2013, Abu Muhammad al-Furqan, and seven others who originally worked for Halab News Network. According to The New York Times, it has a direct connection with IS, from which it "gets tips". Its name was taken from Amik Valley in Hatay Province, which is mentioned in a hadith as the site of an "apocalyptic victory over non-believers". Amaq News Agency was first noticed by SITE during the Siege of Kobanî (Syria) in 2014, when its updates were shared among IS fighters. It became more widely known after it began reporting claims of responsibility for terrorist attacks in Western countries, such as the 2015 San Bernardino attack, for which IS officially claimed responsibility the next day. An Amaq cameraman shot the first footage of the capture of Palmyra in 2015. Amaq launched an official mobile app in 2015 and has warned against unofficial versions that reportedly have been used to spy on its users. It also uses a Telegram account. It had a WordPress-based blog, but it was removed without explanation in April 2016. On 12 June 2016, IS claimed responsibility for the Pulse nightclub shooting through Amaq, without prior knowledge of the attack. The shooter, Omar Mateen had later pledged allegiance to IS via a phone call with emergency services. On 31 May 2017, a Facebook post announced Amaq's founder, Baraa Kadek AKA Rayan Meshaal, had been killed with his daughter by an American airstrike on Mayadin. The post was reportedly made by his younger brother. Reuters could not immediately verify this account. On 27 July 2017, the US confirmed that Kadek had been killed by a coalition airstrike near Mayadin between 25 and 27 May 2017. In June 2017, German police arrested a 23-year-old Syrian man identified only as Mohammed G., accusing him of communicating with the alleged perpetrator of the 2016 Malmö Muslim community centre arson in order to report to Amaq. On 21 March 2019, the U.S. Department of State officially deemed Amaq an alias of IS, and thus a Foreign Terrorist Organization. On 22 March 2024, the Islamic State claimed responsibility for the Crocus City Hall attack through Amaq, U.S. officials confirmed the claim shortly after. A day after the attack, Amaq published a video of the attack, filmed by one of the attackers. It showed the attackers shooting victims and slitting the throat of another, while the filming attacker praises Allah and speaks against infidels. == Character == Amaq publishes a stream of short news reports, both text and video, on the mobile app Telegram. The reports take on the trappings of mainstream journalism, with "Breaking News" headings, and embedded reporters at the scenes of IS battles. The reports try to appear neutral, toning down the jihadist language and sectarian slurs IS uses in its official releases. Charlie Winter of the Transcultural Conflict and Violence Initiative at Georgia State University, and Rita Katz of SITE Intelligence Group in Washington say Amaq functions much like the state-owned news agency of IS, though the group does not acknowledge it as such. Katz said it behaves "like a state media". Amaq appears to have been allowed to develop by IS as a way to have a news outlet that is controlled by the group but is somewhat removed from it, giving IS more of the appearance of legitimacy. == Reliability == According to Rukmini Callimachi in The New York Times: "Despite a widespread view that the Islamic State opportunistically claims attacks with which it has little genuine connection, its track record—minus a handful of exceptions—suggests a more rigorous protocol. At times, the Islamic State has got details wrong, or inflated casualty figures, but the gist of its claims is typically correct." According to Callimachi, the group considers itself responsible for acts carried out by people who were inspired by its propaganda, as well as acts carried out by its own personnel and in some instances, had claimed attacks before the identities of the killers were known. Graeme Wood writing in The Atlantic in October 2017, wrote "The idea that the Islamic State simply scans the news in search of mass killings, then sends out press releases in hope of stealing glory, is false. Amaq may learn details of the attacks from mainstream media ... but its claim of credit typically flows from an Amaq-specific source." An October 2017 article in The Hill, points to two false claims made in the summer of 2017, the Resorts World Manila attack and a false claim that bombs had been planted at Charles de Gaulle Airport in Paris. Also, a claimed IS connection to the 2017 Las Vegas shooting proved to be false. According to Rita Katz on the SITE Intelligence Group website, calling a terrorist a "soldier of the caliphate (warrior from the caliphate)" in a statement issued by Amaq, was the usual way in which IS indicated that it inspired an attack. Centrally coordinated attacks were usually described as "executed by a detachment belonging to the Islamic State", and were often announced by both Amaq and by IS' central media command. == Online presence == In November 2019, Belgian police said they had carried out a successful cyberattack on Amaq, thus leaving IS without an operational communication channel. However, Amaq has since regained online presence, primarily on dark web platforms to make it harder for law enforcement to take them down without physical access to the server hosting the specific platform.
Luma (video)
In video, luma ( Y ′ {\displaystyle Y'} ) represents the brightness in an image (the "black-and-white" or achromatic portion of the image). Luma is typically paired with chroma. Luma represents the achromatic image, while the chroma components represent the color information. Converting R′G′B′ sources (such as the output of a three-CCD camera) into luma and chroma allows for chroma subsampling: because human vision has finer spatial sensitivity to luminance ("black and white") differences than chromatic differences, video systems can store and transmit chromatic information at lower resolution, optimizing perceived detail at a particular bandwidth. == Luma versus relative luminance == Luma is the weighted sum of gamma-compressed R′G′B′ components of a color video—the prime symbols ′ denote gamma compression. The word was proposed to prevent confusion between luma as implemented in video engineering and relative luminance as used in color science (i.e. as defined by CIE). Relative luminance is formed as a weighted sum of linear RGB components, not gamma-compressed ones. Even so, luma is sometimes erroneously called luminance. SMPTE EG 28 recommends the symbol Y ′ {\displaystyle Y'} to denote luma and the symbol Y {\displaystyle Y} to denote relative luminance. === Use of relative luminance === While luma is more often encountered, relative luminance is sometimes used in video engineering when referring to the brightness of a monitor. The formula used to calculate relative luminance uses coefficients based on the CIE color matching functions and the relevant standard chromaticities of red, green, and blue (e.g., the original NTSC primaries, SMPTE C, or Rec. 709). For the Rec. 709 (and sRGB) primaries, the linear combination, based on pure colorimetric considerations and the definition of relative luminance is: Y = 0.2126 R + 0.7152 G + 0.0722 B {\displaystyle Y=0.2126R+0.7152G+0.0722B} The formula used to calculate luma in the Rec. 709 spec arbitrarily also uses these same coefficients, but with gamma-compressed components: Y ′ = 0.2126 R ′ + 0.7152 G ′ + 0.0722 B ′ , {\displaystyle Y'=0.2126R'+0.7152G'+0.0722B',} where the prime symbol ′ denotes gamma compression. == Rec. 601 luma versus Rec. 709 luma coefficients == For digital formats following CCIR 601 (i.e. most digital standard definition formats), luma is calculated with this formula: Y 601 ′ = 0.299 R ′ + 0.587 G ′ + 0.114 B ′ {\displaystyle Y'_{\text{601}}=0.299R'+0.587G'+0.114B'} Formats following ITU-R Recommendation BT. 709 (i.e. most digital high definition formats) use a different formula: Y 709 ′ = 0.2126 R ′ + 0.7152 G ′ + 0.0722 B ′ {\displaystyle Y'_{\text{709}}=0.2126R'+0.7152G'+0.0722B'} Modern HDTV systems use the 709 coefficients, while transitional 1035i HDTV (MUSE) formats may use the SMPTE 240M coefficients: Y 240 ′ = 0.212 R ′ + 0.701 G ′ + 0.087 B ′ = Y 145 ′ {\displaystyle Y'_{\text{240}}=0.212R'+0.701G'+0.087B'=Y'_{\text{145}}} These coefficients correspond to the SMPTE RP 145 primaries (also known as "SMPTE C") in use at the time the standard was created. The change in the luma coefficients is to provide the "theoretically correct" coefficients that reflect the corresponding standard chromaticities ('colors') of the primaries red, green, and blue. However, there is some controversy regarding this decision. The difference in luma coefficients requires that component signals must be converted between Rec. 601 and Rec. 709 to provide accurate colors. In consumer equipment, the matrix required to perform this conversion may be omitted (to reduce cost), resulting in inaccurate color. == Luma and luminance errors == As well, the Rec. 709 luma coefficients may not necessarily provide better performance. Because of the difference between luma and relative luminance, luma does not exactly represent the luminance in an image. As a result, errors in chroma can affect luminance. Luma alone does not perfectly represent luminance; accurate luminance requires both accurate luma and chroma. Hence, errors in chroma "bleed" into the luminance of an image. Note the bleeding in lightness near the borders. Due to the widespread usage of chroma subsampling, errors in chroma typically occur when it is lowered in resolution/bandwidth. This lowered bandwidth, coupled with high frequency chroma components, can cause visible errors in luminance. An example of a high frequency chroma component would be the line between the green and magenta bars of the SMPTE color bars test pattern. Error in luminance can be seen as a dark band that occurs in this area.