Lets talk it out. NP3| Camera | Zoom | Moon

ArnimZola

Has anyone tried it, yet ?

I did, here are the results.

Images on the day after the recent eclipse. (Moon wasn’t red at the time of capture. Apologies for that.)

The moon’s image is replaced in real time, like when you tap to focus on the moon, it would change. And when the picture is clicked, another optimization makes it such.

These are from zoom levels where pro res turns on(two images are on full 60x and one is lower, I don’t remember). I do understand what A.I does, being a data scientist.

But, but, but, hear me out.

There needs to be some sort of filtering to the optimization. I know there might be pre-trained models to which the captured vectors goes through the model to output the optimizations. No doubt they do work well in some general environments.

My solution to this problem would be to, either fine-tune the model with some hand-tuned images of the moon, given the true input. This would greatly leverage the manually tuning variations for the model, but I don’t think this would be in the hands of the team as it might be using some thirtd party proprietary model.

I know previously some samsung phones did it, but they listened and corrected it to an extent that the moon doesn’t look replaced.

I’m not a pro in photography, therefore, I’d love to hear some opinions on it.

However, it was a jaw dropper when I saw the clicked images for a few seconds, till my neurons fired.😆

CMF1_hacker

ArnimZola

CMF1_hacker Yes Exactly this & your dissection of the apk.

Digging more into the thought, I guess, they go to this process because anyway, only one face is visible to anyone on earth except for Australian Continent but its the same face for them as well. Due to the fact that the moon is tidally locked with the earth. Including a timezone & location to get the specific angle image to the user.

I get it . But only acceptable upto an extent. Otherwise the image looks just fake. Or if we would want that, why not just Google the image itself (training data 😏).

They all busy with the NOS4.0, maybe reporting in open beta tests would line this matter up. Hopefully.

Coldpaw

During the video posted we witness a huge change in exposure:

While zoomed out, the automatic exposure mostly orients on the dark sky. Since the moon is actually much brighter for cameras than we might think with our human eye. Once zoomed in, the moon becomes more dominent in the scene and exposure changes - the bright glowing night sun changes into the moon. That’s no AI replacement, that’s simply how cameras work. I have done some very, very amateur-ish moon shots with my mirrorless cameras over the past few years and very well know that kind of behaviour.

What is true is the post processing: Especially for the very extreme zoom levels, AI seems to be hard at work making something more appealing out of the blurry mess. But that’s not only related to moon shots, it happens with every single shot.

Personally I am not sure if AI really replaces the moon or just generates its own rendition based on the scene, as I’ve got some pretty poor examples from my NP3.

CMF1_hacker

Coldpaw there is an AiMoon JNI

I’ll post the code extrapolated from JADX in a second (I just finished a workout am in the changing room).

Edit:

ArnimZola

Coldpaw Thanks for the explanation on the focus part. That explains the changing behaviour of the shot. I didn’t account for the sensitivity to the brightness.

But do checkout the video & apk dissection from @CMF1_hacker as well. It shows the exact spot this matter comes into picture.

Coldpaw

That’s actually kind of disappointing to see. But not too surprising after all, since others doing the same stuff.

It gets back to the question what’s better: Having a blurry image with a hardly recognizable moon or having one which is AI generated - in which case why bother taking a moon shot at all, since it’s generated, one could also simply use some stock image.

CMF1_hacker

Coldpaw

I was looking into this and ran out of time. Is it a file image that is simply employed. Is it a bitmap. It is random. I have fully figured it out yet.

albusoak

On nothing 3a

CMF1_hacker

albusoak yeah, the 3a and 3apro (ASTERIODS and ASTERIODS_PLUS) use the same AiMoon libraries/processes. I don’t mind, otherwise, it’s a bright mess, but it’s nice to understand how it functions.

ArnimZola

Seems like the model doesn’t recognize other shapes of the moon, the gradients doesn’t flow on properly.

It maybe the either the color or shape. Seems like dataset doesn’t contain different shades & shapes of the moon.

The contrast & brightness of the final picture also differs if the shape & color aren’t at a general “threshold”.

So, mostly it looks like a rule based approach, with a small classifier, then the moon image is overlayed & adjusted to the zoom level.

This is not fully AI based or a generative approach.

PS: Speculations.

CMF1_hacker

ArnimZola yeah, you’re correct with the AI misnomer. I selected to state AI based on library name. I stopped digging deeper about the process as I have run out of time, sadly.

Anonymous931

ArnimZola This is not fully AI based or a generative approach.

At 30x zoom it can not take a photo of clear text from just one step away, but it snaps perfectly sharp and clear Moon photo by even fully ignoring cloudy sky. Isn’t that obvious enough?

I mean:

Edit: Just for comparison, with DSLR without going into anything expensive (lens) you need a tripod with timer set on 10 seconds, ISO on lowest, and shutter speed from 1/50 sec to 1/250 sec. And you get this:

Edit 2: Read this thread:

So our little phones with their crappy tiny sensors are smoking DSLR, all without AI? Yeah, no.

CMF1_hacker

Anonymous931 I believe the post you’re replying is commenting on my usage of the term AI and the underlying software processes, not that an underlying process fails to exist.

ArnimZola

Anonymous931 DSLRs or professional cameras might as well get it correct and maybe the image of the moon look the same.

My point is, capturing a moon shot, that replaces the generic moon might add more to the computations. If you look further in the POV, you might get that there’s a room for optimization.

And talking of the zoom levels, the focal point changes on different zoom levels. Meaning the focus of the camera shifts significantly, for a nearby object.

I have achieved some significant sharp images of distant objects after the optimization applies in the gallery, which is not the case of in general zoom levels, specially on a phone camera.

Text is something that doesn’t process well in the best of OCRs & AI, specially at zoomed levels. And that’s why we call telephoto lens for those kind of shots, which do indeed smoke some good cameras out there.

This image is 60x zoomed from NP3, atleast 10ft far from me. The optimization is good on these type of pictures no doubt. But checkout the cable overlap & how the optimizing model is confused between them.

Dushyant_BPpgJ29xmr

you’re still getting great moon photos idk what’s not working for me but the photos I am clicking after putting my NP3(1 week old) on a tripod are just unbelievable. what do you think is the issue.

CMF1_hacker

Dushyant_BPpgJ29xmr your shutter speed is way too long. post your photo app settings.

Dushyant_BPpgJ29xmr

CMF1_hacker what exactly you’re asking for? The camera settings?

CMF1_hacker

Dushyant_BPpgJ29xmr your shutter speed is way too long … how long is your exposure time?

Dushyant_BPpgJ29xmr

CMF1_hacker I didn’t use any specific mode while clicking the photos. I just opened the camera, zoomed in, adjusted the focus, and clicked. The exposure must have been on auto. I was disappointed with the results, so I deleted them. I’ve seen in many reviews that photos of the moon come out well-processed.