Raw histogram and why does it matter

I wonder about the definition. RAW is just a file extension. You can hide a lot under it. If it is just less bit depth file, it is still close to RAW file format. And therefore even engineers might be more likely call it a RAW. Definitions can and DO change, and new are added, if we need to be more specific.

I don't know because I've always directed technical inquiries to the tech support department rather than marketing or engineering.

I have worked for and with a broad assortment of people in marketing positions to supply the images they need of their products. It's generally been a pleasure to do so. Where product images are being generated by an in-house studio, the photographers and studio will generally be part of the marketing department.

it can measure DR

It's a good point to raise. Raw image formats are essentially unprocessed (or minimally processed) files containing the raw capture data needed to process and edit a file for the display or printing of a visible finished image.

The great thing about raw files, compared to film or processed image formats, is that the capture data can be processed and edited any number of times in different ways for different purposes. That said, when you import an image into a conventional raw editing program; a number of basic processing steps have occurred in the background before you see it.

Here's a black frame from a Panasonic S1:

Which shows similar weird "anti-coring" around black. I posted this - or a very similar image - on DPReview a while ago. There was some thought that the notch was caused by some kind of rounding error, but I have trouble imagining a mechanism.

Another weird thing is the exp(-abs(x)) distribution in the flanks.

I tried looking at the red channel with the central half-dozen levels stretched from black to white, and looking at a real FFT of the same thing, but couldn't see anything obvious.

I wondered if something was going wrong at the level of the ADCs.

Were there some way of fixing the "anti-coring", the noise would look quite good.

A (Read-noise * gain + conversion-noise) model doesn't work here, or for CrashpcCZ's camera.

Did you get a copy of that photo, Jim?

David

Thanks. The log scale makes it clearer that the "anti-coring" notch doesn't go all the way dowm, in any channel, for any of the values close to black.

The rest of the distribution is a bit different to what I get on Panasonic S1 - your noise distribution is less symmetric, with a bit of a tail at the high end.

I don't know what that means.

I have no other explanation that the noise is not random. There can be many underlying reasons for that.

Anyways I have neither reason nor tools to diagnose and do anything with such output. 😅

I never even saw it.

That's what I said, yes. That's what Sony said in their definition. And here is what Nikon wrote: "Exclusive to Nikon cameras, the NEF is Nikon's RAW file format. RAW image files, sometimes referred to as digital negatives, contain all the image information captured by the camera's sensor, along with the image's metadata (the camera's identification and its settings, the lens used and other information). The NEF file is written to the memory card in either an uncompressed or "lossless" compressed form. " -- www.nikonusa.com/en/learn-and-explore/a/products-and-innovation/nikon-electronic-format-nef.html

RawDigger opens lossy compressed pseudo-raw files to help researches analyze the differences.
Ciao.

Yes, I wouldn't get too gung-ho about declaring raws as non-images, either. Why reach for the binary definitions, when nuance does a much better job?

I like to think in terms of what happens when we stretch the parameters. For example, if a monitor had enough resolution or you stood far enough away from it, you could actually see the CFA pattern go extinct, whether your display is monochrome or color. So as far as CFA is concerned, it may only be an issue when we are magnifying the pattern too much, just like when we look at a printed image or a monitor screen under a microscope, and see dither patterns or separate little red, green, and blue rectangles; it is just a matter of degree. You could drop 2 out of 3 color channels just like a CFA in each pixel in a quality, realistic sRGB image, and if you step back far enough, the CFA pattern will disappear, and the image will simply seem a little too green and a bit too dark, but varying the brightness of pixels could get around that, and there are many monitors now that can give high pixel output, to compensate.

We could throw a convolution of

0.25 0.25 0.00
0.25 0.25 0.00
0.00 0.00 0.00

on your image, and the CFA effect vanishes, or just swap neighbor pixels randomly, and you get a noise dither instead of a CFA.

Perhaps it would be better to ask how realistic an "alleged" image is, than to ask if it is really an image.

More of what Sony and Nikon have written on the raw compression options they offer...

It's been a while now, so I don't remember the specific options I used, but I once took a variety of lossless Canon raws and converted them to uncompressed DNG, and then ran maximum lossless compression on them as available in 7Zip, and some of the 7Zip-compressed files were as small as 20% the size of the original Canon lossless compressed raw files. That's definitely superior to cRAW, but it is the time element that keeps such compression from being used by cameras; the cameras need to get things done fast, and don't have the time for maximum compression. Also, they take longer to decompress.

I don't remember who or when, but someone did in fact refer to non-ETTR as "under-exposure" in this thread. I know because I started to reply to it, but changed my mind. It should be called "X stops short of ETTR", I think; not "X stops of underexposure", as I think of "underexposure" as a phenomenon of missing the target nominal exposure of an ISO setting, as the default meaning. It is implicitly part of the exposure triangle paradigm, I think, which we can use or ignore.

I wouldn't declare "raws" as non-images, either. They are latent unprocessed images which become visible images when the raw capture data is processed in a manner which renders it in a viewable form.

Not restored, approximated.

That's funny.

LOL

For certain lossy compressed formats white balance is applied to the data.

You appear to have quoted what Nikon has written regarding their NEF raw file format in two different ways. The first is to quote a string of sentences which appear to support your viewpoint. The second chops out words from sentences with little to no recognizable connection to their context. You might consider using longer quotes with meaningful context in the second instance.

Not my viewpoint.
And "more ideal" is simply nonsense. Ideal is an absolute and therefore cannot be compared.