M4T RTK accurate or not ..

[jaja6009]

Thank you for your suggestions. Do you usually use rolling shutter correction with full Txyz and Rxyz parameters when processing data from the Matrice 4T ?

I did the Rolling Shutter correction along with using the suggested Adaptive Camera Model per Agisoft's guidance and did not find it to have changed the outcome in any meaningful way worth reporting back to everyone other than to say I tried it and it did drastically change the outcomes.

I am happy with my processing decisions in Agisfot because I ran the data in both Pix4D and Drone Deploy. Both of these solutions have much less user input, so there are less ways to interfere. With both as we saw in the reported RMSE sheet, Pix4D and Drone Deploy were for the most part very similar in results.

Now to find a day to do the final tests. PPK and I will try to use a Local Base for the best possible outcomes.

My poor M4E also needs to get some flight time. I have been flying the 4T nonstop since I got it. The thermal sensor and IR Cut Filter make night flying a blast.

 

[Mflyer]

I did the Rolling Shutter correction along with using the suggested Adaptive Camera Model per Agisoft's guidance and did not find it to have changed the outcome in any meaningful way worth reporting back to everyone other than to say I tried it and it did drastically change the outcomes.

I am happy with my processing decisions in Agisfot because I ran the data in both Pix4D and Drone Deploy. Both of these solutions have much less user input, so there are less ways to interfere. With both as we saw in the reported RMSE sheet, Pix4D and Drone Deploy were for the most part very similar in results.

Now to find a day to do the final tests. PPK and I will try to use a Local Base for the best possible outcomes.

My poor M4E also needs to get some flight time. I have been flying the 4T nonstop since I got it. The thermal sensor and IR Cut Filter make night flying a blast.

Within your busy schedule.. if you in any way find an opportunity, would you be willing to compare Agisoft numbers to Mipmap? Just curious : )

 

[jaja6009]

Within your busy schedule.. if you in any way find an opportunity, would you be willing to compare Agisoft numbers to Mipmap? Just curious : )

The M4T in Mip Map

All control points were Check Points.

2 of the points are much worse than the others, but I triple checked, they were marked correctly.

I do not have that extreme error on those points in Agisoft Pro.

The Results are very close though to Agisoft in Total RMSE even though this one fared much better in Z.

MipMap



Agisoft

 

[Mflyer]

The M4T in Mip Map

All control points were Check Points.

2 of the points are much worse than the others, but I triple checked, they were marked correctly.

I do not have that extreme error on those points in Agisoft Pro.

The Results are very close though to Agisoft in Total RMSE even though this one fared much better in Z.

MipMap

View attachment 31313

Agisoft

View attachment 31314

Very interesting. Thank you!

 

[jaja6009]

I ran the Matrice 4E in Mip Map.

Double Oblique, No GCPs.

Mip Map




Agisoft Pro

 

[jaja6009]

Interesting results here.

This was RTK. I will do the PPK for the same mission in the future.

It was the school's new baseball field. Again I am in New Jersey and there is not a lot of elevation change, so its limited elevation gain.


RTK Mission
5 Control Points Collected @ 2 minutes each from a base 31 km away. I know too far for RTK.
I made a known point then had the Emlid send corrections to the drone over Local NTRIP. This point was averaged from the 31 km away base for 10 minutes.

All points were used as Check Points.

The M4T was as good as the M4E in almost every test, even beating it using Smart Oblique.

Almost all of my personal tests using the M4E have shown that it is not as accurate as the Mavic 3E we use for beach mapping, hence why I still use the M3E for this purpose. When doing sand volume gain/loss the elevation is the most important aspect with 1 cm of error causing large errors in volume loss/gain.

Unfortunately all of my tests are at long baselines to the base. I am hoping for another test where I have 2 Emlids so that I can just setup on an NGS benchmark nearby and run a test with a very low baseline.

RMSE US S Ft	RMSE cm
M4T Nadir Alt Opt	0.23	0.15	0.52	0.59		6.88	4.72	15.95	18.00
M4T Smart Obl	0.26	0.10	0.14	0.31		7.84	2.90	4.30	9.40
M4E Nadir Alt Opt	0.25	0.07	0.16	0.30		7.52	2.16	4.81	9.18
M4E Smart Obl	0.27	0.08	0.15	0.32		8.11	2.52	4.48	9.60

 

[jaja6009]

Here is the results again, the copy and paste from Excel messed up.

US Survey Ft on the left and cm on the right.

 

[Mflyer]

Smart Oblique outcome is surprising.
Compared in flight time how much does it differ from trad Nadir? The amount of images would times 3 I guess ..

 

[jaja6009]

Images Taken

M4E
Nadir with Alt Opt - 140
Smart Oblique - 152


M4T
Nadir with Alt Opt - 142
Smart Oblique - 211

Please note, I did not use a dedicated KML, so the areas were hand marked and were of differing sizes. But you can gain insight for each drone against itself as far as the flight mode, but do not compare the drones against each other for this test.
For the original main test a dedicated KML was used so you can gain that insight on that test.

 

[Ro3go]

After analysing the flight tests, there's something that still doesn't add up for me in the results of the great work that jaja6009 has done.

First: when we talk about nadir, I assume we're referring to an orthomosaic.

Second: When we talk about smart oblique, the gimbal moves in several directions along the same flight path.

Third: When we talk about oblique, I assume it's when the mission is manually configurable and the 5 steps it performs, the first is nadir and the rest are oblique.

Based on this, I repeated what jaja6009 did using the same parameters: distance/area, altitude, and speed. I programmed the three missions with FlightHub2 to then import them from the cloud in the field.

To my surprise, after reviewing the configurations and starting the first smart oblique mission, I saw that the altitude and speed were no longer the same.

Smart oblique 4.2~15 m/s auto

Nadir 14.2 m/s with elevation optimization

Oblique 14.2 m/s no elevation optimization

I had set it as jaja6009 had done

Alt 63.6, ASL 60.4, which is what the controller showed.

Also, the number of photos per mission is much higher in my case

Oblique photos 326

Smart oblique 469

Nadir 225



Only in the last mission did I modify the speed to what jaja6009 had stipulated [oblique]. Once the missions were finished, I processed the flights with Pix4dmapper, only the first step, and the results were: Quite accurate, see the photo.

Without GCP using NTRIP at a distance of 25km, RTK connection fixed.

Now my question: why are the results so different when the parameters are almost the same? Could geolocation influence the accuracy, or could RTK and satellite connections affect the same mission in different parts of the world, thus causing the M4T's interpretation/opinion to vary depending on the location?

I also have to say that the terrain isn't as flat as jaja6009's, see photos and I'd like to know what parameters appear in the pix4dmapper photo editor. [x,y]

And finally, perhaps other M4T users could replicate what jaj6009 has well done for the good of the community.

 

[jaja6009]

To clear up my tests.

Nadir was only images looking straight down. And I tested with the Altitude Optimization turned both on and off. Altitude Optimization is the last line of oblique images from the outside to the inside.

Oblique was using the Custom Camera option. I used 60 degrees and then flew a double grid by rotating the second mission 90 degrees from the initial mission.

When I changed the parameters I would go back and make sure the altitude was set the same for each mission and also the speed of the drone.

On the question of why the results are so different. You are comparing two different things. I was reporting the RMSE as compared to independently measured control points used as check points, not the math behind the outputs. When I look at the computed image locations on my outputs they too are in the mm range.

The best practice way to assessing absolute accuracy is to validate with check points. The software can come up with whatever values it wants to state for its relative accuracy and how well the outputs line up with the math in the various report sections but only check points tell the true story, the story if the outputs line up to their real locations on the Earth. When the software does not use check points, it is grading itself. With check points you are asking for an independent, second assessment.

Never expect your outputs to be better than the GSD of project with absolute accuracy. You expect 1 to 3 times the GSD. The only time I saw RMSE in the millmeters was with much higher end systems that used PPK, a Total Station to collect control points, using GCPs and a high-end photogrammetry camera that had its lens calibrated by a camera lab.

I have questions on DJI's lens calibration since all software I have used, have the optimized camera calibration not matching DJI's imported, initial calibration. Further when processing using DJI's or letting the software come up with it did not alter the results in the few tests I ran. Alexy from Agisoft always tells users to let Agisoft come up with the initial calibration and not to use DJI's.

 

[jaja6009]

Okay, I finally got around to PPK on the second mission set on the school's new baseball field.

I only processed the Smart Oblique flights.



I applied PPK to the control points and again they were only used as check points.
I then applied PPK to the images.

I used a base from Rock RTK that was about 6 km away, so it was real close.

For me the question of accuracy/precision was answered with the first test I ran, but this cements it for me.

 

[Dave Pitman]

I have questions on DJI's lens calibration since all software I have used, have the optimized camera calibration not matching DJI's imported, initial calibration.

From what I have read, very expensive lenses need to be calibrated over time and even for use when temperature and humidity vary by some margin from the previous calibration. I take DJI's calibration process as more of a quality control procedure at the point of manufacture rather than assuming that the calibration is good to go for the life of the drone. Agisoft's triangulation process likely provides a much more accurate snapshot of the lens parameters at the time of that flight.

 

[jaja6009]

Dave, I read the same thing about the calibrations.
I have tried the various low cost camera calibrations and did not see any results that make me want to continue to use those processes.

I have been wanting to change to PPK workflows, and this may be the final test that makes me do so.
Red Toolbox for the images and Emlid Studio for the control points were very easy to use and did not in anyway make the project more cumbersome. I just need to organize the various files better for the future.