My mother visited last month and wanted to photograph a peacock in our garden. She held up her decade-old point-and-shoot. I held up my phone. Same peacock, same light, same distance. Her photo was blurry and flat. Mine looked like something from a wildlife magazine. Her camera has more optical zoom. Mine has a neural processing unit running 30 frames of computation per second before I even press the button.
The camera hardware in most mid-range smartphones is genuinely average — small sensor, tiny lens, physical limitations that no amount of engineering can fully overcome. The photos they produce are not average. That gap is entirely computational photography: a stack of AI models running silently every time you open the camera app, making hundreds of micro-decisions before you tap the shutter.
Most people think of taking a photo as: point → tap → done. The actual sequence is: point → 30 AI decisions per second → tap → 50 more AI decisions in 0.8 seconds → done.
The moment you open the camera app, your phone begins a continuous processing loop called the viewfinder pipeline. This runs at 30 frames per second and does several things simultaneously: it detects what scene you're looking at, tracks any faces or subjects, measures light levels in different zones of the frame, calculates optimal exposure and white balance, and runs a real-time preview of what the final processed image will look like — not the raw sensor output, but the post-processed version.
This is why modern phone cameras show you a "finished" looking image in the viewfinder rather than the flat, slightly grey raw sensor output. You're already seeing the AI's predicted output before you've committed to the shot. If you switch to a "pro" or "RAW" mode, the viewfinder often looks noticeably worse — that's the honest sensor, without the AI layer.
The first job of the camera AI is to classify what you're pointing at. Modern phones recognise over 100 scene categories: food, pets, sunsets, text documents, QR codes, flowers, mountains, beaches, night sky, fireworks, indoor portraits, outdoor portraits, snow, waterfalls, architecture — and dozens of subcategories within each.
This classification isn't cosmetic. It drives downstream processing decisions:
Exposure is biased toward preserving highlight detail (the sky). Saturation is boosted in the orange/red spectrum. White balance is shifted warmer. HDR aggressiveness is increased.
Saturation significantly increased. Sharpness boosted in the centre. White balance corrected aggressively (food often looks grey under restaurant lighting). Some phones add a very subtle bokeh around the edges automatically.
Face tracking activated (keeps faces in focus priority). Skin tone optimisation turned on. Beauty processing depth set by user preference — smoothing, eye enhancement, skin brightening. Background blur pre-calculated if portrait mode is on.
Perspective correction applied automatically (trapezoidal distortion from an angle is flattened). Contrast maximised. Colour flattened toward black-and-white. Some phones switch to a dedicated document scan mode with OCR.
High Dynamic Range photography solves a fundamental physics problem: the range of light in a real scene — from deep shadow to bright sky — is often far wider than what a sensor can capture in a single exposure. A single shot either blows out the sky or loses the shadows.
HDR on smartphones works by capturing multiple frames at different exposure levels almost simultaneously — one for highlights, one for midtones, one for shadows — and then computationally merging them into a single image that preserves detail across the full range. This happens in under a second, invisibly, every time you take a photo in most lighting conditions.
The hard part isn't capturing the frames — it's merging them without creating "ghosting" on moving subjects. If someone blinks between frames, or a leaf moves in the wind, the frames don't align perfectly. The AI has to detect these inconsistencies and decide which frame to use for each region of the image. A hand that moved between frames gets taken from the sharpest single frame rather than the average.
Google's approach (on Pixel phones) takes this furthest with what they call HDR+ — capturing 15 or more frames in a burst, using the best parts of each, and then applying machine learning to reduce noise and sharpen detail. The result is images with noise characteristics of a much larger sensor than the phone actually has.
Night Mode is where computational photography is most visibly magical. Here's what's actually happening when you hold your phone still for 3–5 seconds in Night Mode:
The camera captures 10–15 frames across the exposure window. Each frame is underexposed (short shutter to avoid blur) but is sharper. The AI then does three things: first, it aligns all the frames precisely — even tiny hand tremor means the frames don't line up pixel-for-pixel, and this alignment is done by a motion estimation model. Second, it merges the frames, adding signal while averaging out random noise (noise is random, so averaging reduces it; real detail is consistent, so it survives). Third, it runs a denoising neural network on the result to clean up any remaining noise, then a sharpening pass to restore fine detail.
What looks like "one long exposure" is actually a neural network averaging 15 short exposures, aligning them in real time, and reconstructing the detail that no single frame could have captured. It's not photography — it's computational reconstruction of a scene.
The result can be genuinely astonishing: a handheld Night Mode shot on a flagship phone in a dark room often surpasses what a professional photographer could get from a DSLR on a tripod a decade ago. The hardware didn't get that good. The AI closed the gap.
Portrait Mode creates a blurred background (bokeh) that optically can only happen with a large lens at a wide aperture — something a phone physically cannot do. So it fakes it.
The subject needs to be separated from the background. Phones with dual cameras use the disparity between the two lenses to construct a rough depth map — areas at different distances from the lens have measurable parallax. Phones with a single camera (increasingly common in mid-range) use a machine learning model trained to estimate depth purely from visual cues: focus gradients, texture, semantic understanding of what's likely to be foreground vs background.
Once the depth map exists, a blur is applied — stronger at greater distances, none on the subject. The hardest part is the subject edge: hair, wisps, glasses frames, earrings. Early portrait modes notoriously struggled with these. Modern ones use a dedicated segmentation model trained specifically on edge cases (literally) to produce clean separations around complex subjects.
Dual cameras made portrait mode better at depth estimation. But since 2019, Google's Pixel phones have produced excellent portrait mode from a single camera, using a neural depth estimation model trained on millions of images. The two-camera approach gives more accurate depth data. A well-trained single-camera model can produce results indistinguishable to most viewers — the limitation is edge accuracy in unusual cases, not the core bokeh effect.
Computational photography has a well-documented bias problem: models trained primarily on lighter-skinned faces historically over-brightened darker skin tones, reducing detail and appearing to "correct" what was never wrong. Indian skin tones across the spectrum from wheatish to deep brown were systematically mishandled by early AI camera systems — faces came out either too bright (losing detail) or with inaccurate colour casts.
Google, Samsung, and Apple have all run dedicated initiatives to address this, with Google's "Real Tone" work being the most publicly documented. They expanded training datasets to include a far wider range of skin tones, changed the accuracy metric used during training (evaluating across the full skin tone spectrum rather than averaging), and specifically tuned Night Mode and Portrait Mode processing for darker skin tones. The gap has significantly narrowed — but reviews of mid-range Android phones from smaller manufacturers still frequently flag skin tone handling as a weakness.
Diwali photography is a genuinely hard computational problem. Fireworks and diyas create extreme point-light sources in darkness — exactly the conditions where HDR merging and Night Mode struggle most. The algorithm has to decide whether those bright points are "highlights to preserve" or "blown-out noise to recover," and the answer changes from frame to frame as fireworks burst and fade. Most phones handle it imperfectly. The phones that do it well have specifically trained their HDR models on fireworks scenes.
Haze and dust — common in North Indian winters and dry-season afternoons — create flat, low-contrast scenes that naive processing turns muddy. Some phones now include specific dehazing models that attempt to reconstruct contrast and colour lost to atmospheric scattering. Results are mixed; the problem is genuinely hard because dehazing from a single frame requires the model to infer what the scene looked like without the haze — which is partly a creative interpretation, not a pure reconstruction.
There's something worth saying plainly: AI camera processing is not always showing you what was there. It's showing you a model's best reconstruction of what was probably there, combined with aesthetic choices baked into training data.
When your phone smooths skin in a portrait, it's not recovering detail — it's removing it and replacing it with an AI's prediction of what smooth skin looks like. When Night Mode brightens a dark scene, the colour in the shadows is partly inferred, not measured. When the food mode boosts saturation, the biryani on your plate didn't get more orange — the AI decided it should look more orange.
For most purposes, this is fine. The photos are beautiful, they capture the moment, they look better than the raw sensor. But professional photographers, journalists, and anyone who cares about photographic accuracy need to know this is happening — and know how to turn it off (RAW mode, or reducing AI processing in Pro mode).
The AI is making your photos look better by its definition of "better." That definition was trained on millions of images judged by human raters. It's a reasonable definition. It's not the only one.
After the peacock incident, I went into Pro mode on my phone and took the same shot without AI processing. The result was honestly closer to my mother's point-and-shoot: correct, flat, slightly noisy, clearly taken with a small sensor at distance. The peacock's blue-green feathers were there but muted.
Then I switched back to Auto. The AI-processed version had vivid, almost electric blues and greens in the plumage, sharp detail in the eye, and a subtly blurred background. It looked incredible. It also looked better than what I actually saw with my own eyes standing in the garden.
Both photos are "true." One is what the sensor captured. The other is what the AI thought would make a better photograph. For sharing and memories, I'll take the AI version every time. For anything that requires documentary accuracy, the Pro mode version is the honest one.
Next — the final article in this series: AI in your smartphone keyboard. Autocorrect, next-word prediction, swipe typing, and the model that's learned to predict not just your spelling but your exact phrasing.