Deep image prior

Deep image prior

Deep image prior is a type of convolutional neural network used to enhance a given image with no prior training data other than the image itself. A neural network is randomly initialized and used as prior to solve inverse problems such as noise reduction, super-resolution, and inpainting. Image statistics are captured by the structure of a convolutional image generator rather than by any previously learned capabilities. == Method == === Background === Inverse problems such as noise reduction, super-resolution, and inpainting can be formulated as the optimization task x ∗ = m i n x E ( x ; x 0 ) + R ( x ) {\displaystyle x^{}=min_{x}E(x;x_{0})+R(x)} , where x {\displaystyle x} is an image, x 0 {\displaystyle x_{0}} a corrupted representation of that image, E ( x ; x 0 ) {\displaystyle E(x;x_{0})} is a task-dependent data term, and R(x) is the regularizer. Deep neural networks learn a generator/decoder x = f θ ( z ) {\displaystyle x=f_{\theta }(z)} which maps a random code vector z {\displaystyle z} to an image x {\displaystyle x} . The image corruption method used to generate x 0 {\displaystyle x_{0}} is selected for the specific application. === Specifics === In this approach, the R ( x ) {\displaystyle R(x)} prior is replaced with the implicit prior captured by the neural network (where R ( x ) = 0 {\displaystyle R(x)=0} for images that can be produced by a deep neural networks and R ( x ) = + ∞ {\displaystyle R(x)=+\infty } otherwise). This yields the equation for the minimizer θ ∗ = a r g m i n θ E ( f θ ( z ) ; x 0 ) {\displaystyle \theta ^{}=argmin_{\theta }E(f_{\theta }(z);x_{0})} and the result of the optimization process x ∗ = f θ ∗ ( z ) {\displaystyle x^{}=f_{\theta ^{}}(z)} . The minimizer θ ∗ {\displaystyle \theta ^{}} (typically a gradient descent) starts from a randomly initialized parameters and descends into a local best result to yield the x ∗ {\displaystyle x^{}} restoration function. ==== Overfitting ==== A parameter θ may be used to recover any image, including its noise. However, the network is reluctant to pick up noise because it contains high impedance while useful signal offers low impedance. This results in the θ parameter approaching a good-looking local optimum so long as the number of iterations in the optimization process remains low enough not to overfit data. === Deep Neural Network Model === Typically, the deep neural network model for deep image prior uses a U-Net like model without the skip connections that connect the encoder blocks with the decoder blocks. The authors in their paper mention that "Our findings here (and in other similar comparisons) seem to suggest that having deeper architecture is beneficial, and that having skip-connections that work so well for recognition tasks (such as semantic segmentation) is highly detrimental." == Applications == === Denoising === The principle of denoising is to recover an image x {\displaystyle x} from a noisy observation x 0 {\displaystyle x_{0}} , where x 0 = x + ϵ {\displaystyle x_{0}=x+\epsilon } . The distribution ϵ {\displaystyle \epsilon } is sometimes known (e.g.: profiling sensor and photon noise) and may optionally be incorporated into the model, though this process works well in blind denoising. The quadratic energy function E ( x , x 0 ) = | | x − x 0 | | 2 {\displaystyle E(x,x_{0})=||x-x_{0}||^{2}} is used as the data term, plugging it into the equation for θ ∗ {\displaystyle \theta ^{}} yields the optimization problem m i n θ | | f θ ( z ) − x 0 | | 2 {\displaystyle min_{\theta }||f_{\theta }(z)-x_{0}||^{2}} . === Super-resolution === Super-resolution is used to generate a higher resolution version of image x. The data term is set to E ( x ; x 0 ) = | | d ( x ) − x 0 | | 2 {\displaystyle E(x;x_{0})=||d(x)-x_{0}||^{2}} where d(·) is a downsampling operator such as Lanczos that decimates the image by a factor t. === Inpainting === Inpainting is used to reconstruct a missing area in an image x 0 {\displaystyle x_{0}} . These missing pixels are defined as the binary mask m ∈ { 0 , 1 } H × V {\displaystyle m\in \{0,1\}^{H\times V}} . The data term is defined as E ( x ; x 0 ) = | | ( x − x 0 ) ⊙ m | | 2 {\displaystyle E(x;x_{0})=||(x-x_{0})\odot m||^{2}} (where ⊙ {\displaystyle \odot } is the Hadamard product). The intuition behind this is that the loss is computed only on the known pixels in the image, and the network is going to learn enough about the image to fill in unknown parts of the image even though the computed loss doesn't include those pixels. This strategy is used to remove image watermarks by treating the watermark as missing pixels in the image. === Flash–no-flash reconstruction === This approach may be extended to multiple images. A straightforward example mentioned by the author is the reconstruction of an image to obtain natural light and clarity from a flash–no-flash pair. Video reconstruction is possible but it requires optimizations to take into account the spatial differences. == Implementations == A reference implementation rewritten in Python 3.6 with the PyTorch 0.4.0 library was released by the author under the Apache 2.0 license: deep-image-prior A TensorFlow-based implementation written in Python 2 and released under the CC-SA 3.0 license: deep-image-prior-tensorflow A Keras-based implementation written in Python 2 and released under the GPLv3: machine_learning_denoising == Example == See Astronomy Picture of the Day (APOD) of 2024-02-18

Hallucination (artificial intelligence)

In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation, or delusion) is a response generated by AI that contains false or misleading information presented as fact. This term draws a loose analogy with human psychology, where a hallucination typically involves false percepts. For example, a chatbot powered by large language models (LLMs), like ChatGPT, may embed plausible-sounding random falsehoods within its generated content. Detecting and mitigating errors and hallucinations pose significant challenges for practical deployment and reliability of LLMs in high-stakes scenarios, such as chip design, supply chain logistics, and medical diagnostics. Some software engineers and statisticians have criticized the specific term "AI hallucination" for unreasonably anthropomorphizing computers. Symbolic artificial intelligence models generally do not produce hallucinations, unlike large language models. == Term == === Origin === Since the 1980s, the term "hallucination" has been used in computer vision with a positive connotation to describe the process of adding detail to an image. For example, the task of generating high-resolution face images from low-resolution inputs is called face hallucination. The first documented use of the term "hallucination" in this sense is in the PhD thesis of Eric Mjolsness in 1986. A notable work is the face hallucination algorithm by Simon Baker and Takeo Kanade published in 1999. In the 2000s, hallucinations were described in statistical machine translation as a failure mode. Since the 2010s, the term has undergone a semantic shift to signify the generation of factually incorrect or misleading outputs by AI systems in tasks like machine translation and object detection. In 2015, hallucinations were identified in visual semantic role labeling tasks by Saurabh Gupta and Jitendra Malik. In 2015, computer scientist Andrej Karpathy used the term "hallucinated" in a blog post to describe his recurrent neural network (RNN) language model generating an incorrect citation link. In 2017, Google researchers used the term to describe the responses generated by neural machine translation (NMT) models when they are not related to the source text, and in 2018, the term was used in computer vision to describe instances where non-existent objects are erroneously detected because of adversarial attacks. In July 2021, Meta warned during its release of BlenderBot 2 that the system is prone to "hallucinations", which Meta defined as "confident statements that are not true". Following OpenAI's ChatGPT release in beta version in November 2022, some users complained that such chatbots often seem to pointlessly embed plausible-sounding random falsehoods within their generated content. Many news outlets, including The New York Times, started to use the term "hallucinations" to describe these models' frequently incorrect or inconsistent responses. In 2023, the Cambridge dictionary updated its definition of hallucination to include this new sense specific to the field of AI. Some researchers have highlighted a lack of consistency in how the term is used, but also identified several alternative terms in the literature, such as confabulations, fabrications, and factual errors. === Definitions and alternatives === Uses, definitions and characterizations of the term "hallucination" in the context of LLMs include: "a tendency to invent facts in moments of uncertainty" (OpenAI, May 2023) "a model's logical mistakes" (OpenAI, May 2023) "fabricating information entirely, but behaving as if spouting facts" (CNBC, May 2023) "making up information" (The Verge, February 2023) "probability distributions" (in scientific contexts) Journalist Benj Edwards, in Ars Technica, writes that the term "hallucination" is controversial, but that some form of metaphor remains necessary; Edwards suggests "confabulation" as an analogy for processes that involve "creative gap-filling". In July 2024, a White House report on fostering public trust in AI research mentioned hallucinations only in the context of reducing them. Notably, when acknowledging David Baker's Nobel Prize-winning work with AI-generated proteins, the Nobel committee avoided the term entirely, instead referring to "imaginative protein creation". Hicks, Humphries, and Slater, in their article in Ethics and Information Technology, argue that the output of LLMs is "bullshit" under Harry Frankfurt's definition of the term, and that the models are "in an important way indifferent to the truth of their outputs", with true statements only accidentally true, and false ones accidentally false. Some researchers also use the derogatory term "botshit", often referring to uncritical use of AI. === Criticism === In the scientific community, some researchers avoid the term "hallucination", seeing it as potentially misleading. It has been criticized by Usama Fayyad, executive director of the Institute for Experimental Artificial Intelligence at Northeastern University, on the grounds that it misleadingly personifies large language models and is vague. Mary Shaw said, "The current fashion for calling generative AI's errors 'hallucinations' is appalling. It anthropomorphizes the software, and it spins actual errors as somehow being idiosyncratic quirks of the system even when they're objectively incorrect." In Salon, statistician Gary Smith argues that LLMs "do not understand what words mean" and consequently that the term "hallucination" unreasonably anthropomorphizes the machine. Murray Shanahan argues that anthropomorphic framing of LLM capabilities, including terms like "hallucination", encourages users and researchers to attribute cognitive processes to systems that operate through statistical pattern completion, and advocates for more careful linguistic practices when discussing LLM behavior. Kristina Šekrst argues that applying psychological vocabulary to LLM outputs obscures the difference between the appearance of mental properties and their genuine presence. Förster & Skop assert that tech companies use the hallucination metaphor to anthropomorphize models and deflect responsibility for non-factual outputs. Some see the AI outputs not as illusory but as prospective—that is, having some chance of being true, similar to early-stage scientific conjectures. The term has also been criticized for its association with psychedelic drug experiences. == In natural language generation == In natural language generation, there are several reasons why natural language models hallucinate: === Hallucination from data === Hallucinations can stem from incomplete, inaccurate or unrepresentative data sets. === Modeling-related causes === The pre-training of generative pretrained transformers (GPT) involves predicting the next word. It incentivizes GPT models to "give a guess" about what the next word is, even when they lack information. Some researchers take an anthropomorphic perspective and posit that hallucinations arise from a tension between novelty and usefulness. For instance, Amabile and Pratt define human creativity as the production of novel and useful ideas. By extension, a focus on novelty in machine creativity can lead to the production of original but inaccurate responses—that is, falsehoods—whereas a focus on usefulness may result in memorized content lacking originality. By 2022, newspapers such as The New York Times expressed concern that, as the adoption of bots based on large language models continued to grow, unwarranted user confidence in bot output could lead to problems. === Interpretability research === In 2025, interpretability research by Anthropic on the LLM Claude identified internal circuits that cause it to decline to answer questions unless it knows the answer. By default, the circuit is active and the LLM doesn't answer. When the LLM has sufficient information, these circuits are inhibited and the LLM answers the question. Hallucinations were found to occur when this inhibition happens incorrectly, such as when Claude recognizes a name but lacks sufficient information about that person, causing it to generate plausible but untrue responses. === Examples === On 15 November 2022, researchers from Meta AI published Galactica, designed to "store, combine and reason about scientific knowledge". Content generated by Galactica came with the warning: "Outputs may be unreliable! Language Models are prone to hallucinate text." In one case, when asked to draft a paper on creating avatars, Galactica cited a fictitious paper from a real author who works in the relevant area. Meta withdrew Galactica on 17 November due to offensiveness and inaccuracy. OpenAI's ChatGPT, released in beta version to the public on November 30, 2022, was based on the foundation model GPT-3.5 (a revision of GPT-3). Professor Ethan Mollick of Wharton called it an "omniscient, eager-to-please intern who sometimes lies to you". Data scientist Teresa Kuba

Pixelmator Pro

Pixelmator Pro is a photo, video, and vector graphic editor developed by Apple for macOS and iPadOS as part of its Pixelmator and pro apps platforms and as a part of their Apple Creator Studio suite of applications. Pixelmator Pro relies heavily on technologies from Apple platforms such as Metal, CoreML, Core Image, AVFoundation, GCD, and SwiftUI. == Features == GPU accelerated with Metal 50+ standard image editing tools Layer-based image editor Video editing support Vector graphic support (including SVG support) AI-powered editing features such as background removal ML Super Resolution and Smart Replace Supports a variety of media formats (JPEG, RAW, Apple ProRAW, PSD, PNG, GIF, MP4, HEIF, etc) == Reception == Pixelmator Pro was generally well-received by reviewers who praised its deep use of machine learning, fully macOS-native design, and relatively affordable one-time purchase compared to subscription software such as Adobe Photoshop. Some reviewers criticized that some features are hard to find or hard to use. It was awarded Apple's Mac App of the Year in 2018. Pixelmator Pro does not have support for panorama stitching. == Acquisition by Apple == On November 1, 2024, the Pixelmator Team announced that they were to be acquired by Apple, subject to regulatory approval. Their site promises "There will be no material changes to the Pixelmator Pro, Pixelmator for iOS, and Photomator apps at this time." The acquisition was completed in February 2025. On January 13, 2026, Apple announced that a new version of Pixelmator Pro with AI features would be included in its new Apple Creator Studio subscription, the app would be brought to the iPad and the Mac app would be redesigned with Liquid Glass. == Version history == == Applescript == In 2020 Pixelmator Pro added the ability to leverage Apple's automation language 'AppleScript' to automate many tasks in version 1.8 (Lynx). This enabled simple and advanced automation activities such as image resize, crop, color adjustments, format change, moving layers around, and more advanced actions like removing background, Gaussian blur, text replacement, shadows, color replacement, etc.

AVS Video Editor

AVS Video Editor is a video editing software published by Online Media Technologies Ltd. It is a part of AVS4YOU software suite which includes video, audio, image editing and conversion, disc editing and burning, document conversion and registry cleaner programs. It offers the opportunity to create and edit videos with a vast variety of video and audio effects, text and transitions; capture video from screen, web or DV cameras and VHS tape; record voice; create menus for discs, as well as to save them to plenty of video file formats, burn to discs or publish on Facebook, YouTube, Flickr, etc. == Description == === Interface === The layout consists of the timeline or storyboard view, preview pane and media library (transitions, video effects, text or disc menus) collections. The storyboard view shows the sequence of video clips with the transitions between them and used to change the order of clips or add transitions. Timeline view consists of main video, audio, effects, video overlay and text lines for editing. Once on the timeline video can be duplicated, split, muted, frozen, cropped, stabilized, its speed can be slowed down or increased, audio and color corrected. === Importing footage === Video, audio and image files necessary for video project can be imported into the program from computer hard disk drive. User can also capture video from computer screen, web or mini DV camera, as well as from VHS tape, record voice. === Output (web, device, disc, format) === AVS Video Editor gives the opportunity to save video to a computer hard drive to one of the video formats: AVI, DVD, Blu-ray, MOV, MP4, M4V, MPEG, WMV, MKV, WebM, M2TS, TS, FLV, SWF, RM, 3GP, GIF, DPG, AMV, MTV; burn to DVD or Blu-ray disc with menus; create a video for mobile players, mobile phones or gaming consoles and upload it right to the device. The most popular devices such as Apple iPod, Apple iPhone, Apple iPad, Sony PSP, Samsung Galaxy, Android and BlackBerry smartphones and tablets are supported. There is also an option to create a video that can be streamed via web and save it into Flash or WebM format or for the popular web services: YouTube, Facebook, Telly (Twitvid), Dailymotion, Flickr and Dropbox. === Features === Single and multithread modes: if a computer supports multi-threading, video creation process is performed faster in multithread mode, especially on a multi-core system. Customization of the output file settings, such as bitrate, frame rate, frame size, video and audio codecs, etc. Transitions - help video clips smoothly go into one another, dissolve or overlap two video or image files. Fade in and fade out video and audio files - dissolve a video to and from a blank image, reduce the audio volume at the end of the video and increase at the beginning. Slideshow creation - create a presentation of a series of still images. Voice recording Projects - once a project is created and saved, the next time saving video to some other format will be fast, projects are also used if a user do not have a possibility to create, edit and save video all at once. Video overlay option - superpose video image over the video clip that is being edited. Disk menu and chapters creation - an option for DVD and Blu-ray video. Freeze frame - make a still shot from a video clip. Stabilization feature - reduce jittering or blurring caused by shaky motions of a camera. Enhanced deinterlacing method - increase video quality for interlaced input file - spots and blurred areas are compensated. Scene detection - search and separate one scene of the video from the other. Loop DVD and SWF - output SWF and DVD video are played back continuously. Caching for processing high definition files - create a duplicate video file smaller in size to use it on the preview window and accelerate processing of HD files. Chroma key option - add video overlay half transparent so that only part of it is visible and all the rest disappears to reveal the video underneath. Capture video material from DV tapes, VHS tapes, web cameras, etc. Movie closing credits - add information on movie editing, e.g. crew, cast, data, etc. Creeping line, subtitles, text - add different captions (static and animated), shapes and images to video. Speech balloons and other graphic objects - geometrical shapes to highlight an object in the video. Zoom effect - magnify or reduce the view of the image. Rotate effect - rotate video image at different degrees, e.g. 90, 180, etc. Grayscale and old movie effects - create a black and white video image. Old movie adds also scratches, noise, shake and dust to video, as if it's being played on an old projector. Blur and sharpen effects - visually smooth and soften an image, or make video image better focused. Snow and particles effects - adds snow or various objects (bubbles, flowers, leaves, butterflies etc.) that are moving, flying or falling on the video. Pan and zoom Timer, countdown effects - add a timepiece that measures or counts down a time interval to the video being edited. Snapshots - capture a particular moment of a video clip. Sound track replacement - mute audio track from video and add another one. Audio amplify, noise removal, equalizer, etc. - make video sound louder, attenuate the noise, change frequency pattern of the audio, make some other audio adjustments. Trim and multi-trim options - change video clip duration cutting out unnecessary parts or detect scenes and cut out parts in any place of the video clip. Color correction (brightness, temperature, contrast, saturation, gamma, etc.) effects - allow adjustment of tonal range, color, and sharpness of video files. Crop scale effect - get rid of mattes that appear after changing aspect ratio of a video file. Adjusting the Playback Speed Volume and balance - change sound volume in the output video. Change volume value proportion for main video and added soundtrack, completely mute main video audio and leave added soundtrack only, etc. === Utilities embedded into AVS Video Editor === AVS Mobile Uploader is used to transfer edited and converted media files to portable devices via Bluetooth, Infrared or USB connection. AVS Video Burner is used to burn converted video files to different disc types: CD, DVD, Blu-ray. AVS Video Recorder is used to capture video from analog video sources and supports different types of devices: capture card, web camera (webcam), DV camera, HDV camera. AVS Video Uploader is used to transfer video files to popular video-sharing websites, like Facebook, Dailymotion, YouTube, Photobucket, TwitVid, MySpace, Flickr. AVS Screen Capture is used to capture any actions on the desktop to make presentations or video tutorials more vivid and easily comprehensible. == Important upgrades == The initial release of AVS Video Editor was in 2003 when the program was offered inside AVS software bundles together with AVS Video Tools, AVS Audio Tools and DVD Copy software. In 2005 the program is offered as a part of multifunctional AVS4YOU software suite. AVS Video Editor is frequently updated. The main updates include adding several important features for video editing

Cooliris (plugin)

Cooliris (for Desktop), formerly known as PicLens, was a web browser extension developed by Cooliris, Inc, and later acquired by Yahoo. The plugin provides an interactive 3D-like experience for viewing digital images and videos from the web and from desktop applications. The software places a small icon atop image thumbnails that appear on a webpage. Clicking on the icon loads the Cooliris 3D Wall, a browsing environment that gives the user the effect of flying through a three-dimensional space. Released to the public in January 2008, The New York Times described Cooliris as the "new immersive approach to Web navigation". Cooliris went out to win the 2008 Crunchies Award for Best Design. The plugin has received over 50 million downloads. As of May 2014 browser plugins are unavailable from the official website. There are only links to tablet apps - for iOS and Android.

Oculus Medium

Oculus Medium is a digital sculpting software that works with virtual reality headsets and 6DoF motion controllers. It is used to create and paint digital sculptures. Medium works only on Oculus Rift. It was released on December 5, 2016, following with a major update in 2018 introducing new features and a revamped UI. On December 9, 2019, Oculus Medium was acquired by Adobe and re-named to "Medium by Adobe".

YrWall

YrWall is a Digital Graffiti Wall developed by event company Luma, where designs are created on a large wall using a modified spray paint can. The can contains no paint, instead it has an IR light which is tracked by a computer vision system and the image immediately back-projected onto the wall. The inbuilt YrWall software has much of the functionality of a typical computer paint program, with a pop-out interface which enables users to change colour, spray width, opacity, work with stencils and use animated items such as swirls, stars, drips and splats. Recent additions to YrWall include options to email a JPEG of the completed design and create personalised stickers and T-shirts. == Dragons' Den == The inventor of YrWall, Tom Hogan, and his business partner, Tim Williams, appeared on Episode 4 of Series 8 of the BBC show Dragons' Den. Seeking investment in YrWall, the entrepreneurs were successful in gaining £50,000 for 40% of the YrWall parent company Lumacoustics from Dragons Deborah Meaden and Peter Jones. == World's Largest Interactive Graffiti Wall == In September 2009 YrWall was used to create the 'World's Largest Interactive Graffiti Wall' at the Bristol Festival, UK. Artists used the standard 3.5 m2 YrWall to produce artwork which was in turn projected live onto a 26m x 10m space on the side of the iconic Lloyds amphitheatre building.