With all these tests, im really curious what speed, iso and shutter speed is used. The 4t must in any way compensate for rolling shitter. Please compare this as well in any tests.
M4T RTK accurate or not ..
- Thread starter Mflyer
- Start date
I plan on the following for my test.
The drone should be here on Wednesday and hopefully within a week I can find time to do a test.
Drones in test.
Matrice 4E
Matrice 4T
Mavic 2 Enterprise Advanced
Mavic 3E
Phantom 4 RTK
All drones will receive corrections from my RTK subscription service located 19 km away.
All drones will import a kml that will then be used to plan a single grid flight plan with Altitude Optimization turned on (Oblique images from the outside to the center of the map).
Overlap will be 80/80
GSD will be set at 1cm so the altitudes will differ due to the different camera specs/focal lengths.
All drones will fly at a maximum of 15 mph.
No GCPs will be used.
5 Check Points will be laid out evenly and measured with an Emlid RS2+ receiving corrections from my RTK service. Each point will be collected for 2 minutes.
I will process in Agisoft Metashape Pro. I can also process in Drone Deploy, Pix4Dmapper, Pix4Dmatic and MipMap.
Datums and Coordinate Systems: Drone: NAD83(2011) with ellipsoid elevations. GNSS Rover: NAD83(2011) with ellipsoid elevations. Agisoft outputs: NAD83(2011) NJ State Plane US Survey ft, orthometric elevations in NAVD88 US Survey ft using Geoid 18.
I can process the data for the drones lacking a mechanical shutter two times. One time without Rolling Shutter Compensation and the second time with Rolling Shutter Compensation if needed.
I may have one wildcard GCP in the center of the map to test if using just that one GCP increases accuracy/precision, especially in the vertical.
When the tests are completed, I will post the Quality Report showing the RMSE for the Check Points.
If I had to predict a winner I would bet on the Mavic 3E. But the Phantom 4 RTK has always been a rock solid mapping drone. It just has longer intervals between images slowing it down and its images are not as good as the Matrice 4E and Mavic 3E. In my testing of mapping similar areas between the M3E, P4RTK and M4E, the Matrice 4E has yet to consistently beat the Phantom 4 RTK or Mavic 3E, most likely due to me always maxing out the speed on the Matrice 4E since it can take and write images so fast and it still not being fully dialed in by DJI.
My predictions:
1. M3E
2. P4RTK
3. M4E
4. M4T
5. M2EA (RMSE measured in meters in the horizontal)
I do have access to an M300RTK at work, we do not have the P1 camera though. I wish I could add this full frame camera to the mix.
Please feel free to critique this test.
The drone should be here on Wednesday and hopefully within a week I can find time to do a test.
Drones in test.
Matrice 4E
Matrice 4T
Mavic 2 Enterprise Advanced
Mavic 3E
Phantom 4 RTK
All drones will receive corrections from my RTK subscription service located 19 km away.
All drones will import a kml that will then be used to plan a single grid flight plan with Altitude Optimization turned on (Oblique images from the outside to the center of the map).
Overlap will be 80/80
GSD will be set at 1cm so the altitudes will differ due to the different camera specs/focal lengths.
All drones will fly at a maximum of 15 mph.
No GCPs will be used.
5 Check Points will be laid out evenly and measured with an Emlid RS2+ receiving corrections from my RTK service. Each point will be collected for 2 minutes.
I will process in Agisoft Metashape Pro. I can also process in Drone Deploy, Pix4Dmapper, Pix4Dmatic and MipMap.
Datums and Coordinate Systems: Drone: NAD83(2011) with ellipsoid elevations. GNSS Rover: NAD83(2011) with ellipsoid elevations. Agisoft outputs: NAD83(2011) NJ State Plane US Survey ft, orthometric elevations in NAVD88 US Survey ft using Geoid 18.
I can process the data for the drones lacking a mechanical shutter two times. One time without Rolling Shutter Compensation and the second time with Rolling Shutter Compensation if needed.
I may have one wildcard GCP in the center of the map to test if using just that one GCP increases accuracy/precision, especially in the vertical.
When the tests are completed, I will post the Quality Report showing the RMSE for the Check Points.
If I had to predict a winner I would bet on the Mavic 3E. But the Phantom 4 RTK has always been a rock solid mapping drone. It just has longer intervals between images slowing it down and its images are not as good as the Matrice 4E and Mavic 3E. In my testing of mapping similar areas between the M3E, P4RTK and M4E, the Matrice 4E has yet to consistently beat the Phantom 4 RTK or Mavic 3E, most likely due to me always maxing out the speed on the Matrice 4E since it can take and write images so fast and it still not being fully dialed in by DJI.
My predictions:
1. M3E
2. P4RTK
3. M4E
4. M4T
5. M2EA (RMSE measured in meters in the horizontal)
I do have access to an M300RTK at work, we do not have the P1 camera though. I wish I could add this full frame camera to the mix.
Please feel free to critique this test.
Drone paradise in here..
Thanks for your well explained plan. As this is to validate m4t accuracy, Herewith my following suggestions.
- Route speed. Can you test the M4t with various speeds and shutter speed. As mentioned, my guess is that speed will influence the accuracy alot as its electronic shutter.
- Software. MipMap sounds and looks very interesting. Could you compare 1 package as industry standard against MipMap for processing. Which software has the shutter compensation?
-GCP. In this case the checkpoints will be sufficient and the GCP will surely increase accuracy as it is taken into account during processes. If you want to see how it compares, sure but its not needed for the initial test.
Thanks in advance for your time and testing!
Thanks for your well explained plan. As this is to validate m4t accuracy, Herewith my following suggestions.
- Route speed. Can you test the M4t with various speeds and shutter speed. As mentioned, my guess is that speed will influence the accuracy alot as its electronic shutter.
- Software. MipMap sounds and looks very interesting. Could you compare 1 package as industry standard against MipMap for processing. Which software has the shutter compensation?
-GCP. In this case the checkpoints will be sufficient and the GCP will surely increase accuracy as it is taken into account during processes. If you want to see how it compares, sure but its not needed for the initial test.
Thanks in advance for your time and testing!
Last edited:
, there's no room for criticism, but there is room to thank you for the time you've taken to teach us and share your knowledge. I wish I could do all those tests!Mflyer, on this test I may not get a chance with multiple speeds on the 4T, but you can be certain that I will be testing the 4T in mapping and modeling constantly, so I will have this information in the near future. I chose 15 mph since it should help the 4T with Rolling Shutter distortion.
I am processing on Agisoft Metashape first since it it is my go to photogrammetry software. Once Agisoft finishes, I will process in Pix4D, Drone Deploy and then MipMap.
In my experience, Agisoft is better of the three, but others may not agree.
I am only going to use Check Points in the initial processing. I will use the single GCP, in the future to see what it does to each output.
Mip Map is being used mostly for curiosity, but also for future tests on 3D models and Mip Map being a simple solution for Gaussian Splatting.
My 4T is actually delayed right right now, but hopefully gets by US Customs. I am hoping for this weekend or next week for the tests.
I am processing on Agisoft Metashape first since it it is my go to photogrammetry software. Once Agisoft finishes, I will process in Pix4D, Drone Deploy and then MipMap.
In my experience, Agisoft is better of the three, but others may not agree.
I am only going to use Check Points in the initial processing. I will use the single GCP, in the future to see what it does to each output.
Mip Map is being used mostly for curiosity, but also for future tests on 3D models and Mip Map being a simple solution for Gaussian Splatting.
My 4T is actually delayed right right now, but hopefully gets by US Customs. I am hoping for this weekend or next week for the tests.
Update.
I have received the drone. Wow, what a great thermal solution. The thermal image, while being the same resolution as my Mavic 2 Enterprise Advanced, definitely has software enhancement and possibly is an upgraded sensor, especially with the Ultra/Super Resolution enabled.
We are still having winter weather, and it has been too windy to take the drone out for full flight testing.
I have already programmed the test flights on all of the drones for the test, but the weather will be a factor. I wanted to do it tomorrow, but wind gusts will be almost 30 mph.
The upcoming week does not have any promising days either for times that I am off from work.
Unfortunately, the best possible day I see in the near future is Wednesday, the 25th.
Here is how the test will be conducted, and I took people's ideas and implemented them the best I could.
Test name - DJI Small Form RTK Drone Shootout
Drones:
Matrice 4T
Matrice 4E
Mavic 3E
Phantom 4RTK
Mavic 2 Enterprise Advanced
Test Area
4.14 acres or 17,765 sq. meters
Mostly flat with elevation changes from 6 to 9 feet MSL.
Flight Altitude and GSD
I am not going to base the test on GSD as that would have 2 of the drones fly way too low. I will have all drones fly at 152 feet AGL. Two of the drones will have a greater GSD because of this, but it will still be under 2cm.
Flight Speed
The Phantom 4 RTK and Mavic 2 Enterprise Advanced will fly slower than the other drones since they cannot take images as fast as the newer solutions. The Mavic 3E and both Matrices will be limited to 15mph. I also will fly these same 3 drones at their maximum speed in separate flights as part of the test.
Nadir and Oblique
All drones will fly a nadir flight path with Altitude Optimization enabled and then fly another mission with oblique double grid. Nadir and oblique imaged flights will be processed by themselves and be part of the comparison test.
Overlap
Nadir - 80/80
Oblique - 75/75
Control Points
The first tests will have NO GCP's and use 5 check points. A final test will use a single GCP to see how much that helps in increasing absolute accuracy. All control points will be collected with an Emlid RS2+ or RS3 that will be receiving corrections from my RTK subscription service.
Drone RTK
All drones will receive corrections from my RTK subscription service. The base unit is located 19km away. I will pause any mission where FIX is lost and will only continue with FIX. If this is not possible I may use Red Toolbox to PPK all data.
Data Datums/Coordinate Systems
The Emlid GNSS rovers will collect in geographic Nad 83 (2011) with ellipsoid elevations and also in projection Nad83 (2011) New Jersey State plane US Survey feet, and orthometric elevations in NAVD88 US Survey feet using Geoid 18.
All drones will receive corrections in geographic Nad 83 (2011) with ellipsoid elevations since that this what the base is sending in.
All processing in all software will output in projection Nad 83 (2011), New Jersey State Plane US Survey feet with orthometric elevations in NAVD88 US Survey feet using Geoid 18.
Software
The main tests will be processed on Agisoft Metashape Pro. I will share the outputs on Agi Cloud or maybe even Arc Online.
On Agisoft, I will use the USGS workflow to remove questionable points on the sparse point cloud before final optimization.
Agisoft will output: 2D ortho map, 2D DEM, 3D Tiled Model
I will also process on Drone Deploy since it is a simple upload and walk away process. I will share the Drone Deploy page with all maps as layers.
In the future I will process in Pix4Dmapper, Pix4Dmatic and Mip Map.
I will use Mip Map's Gaussian Splatting for oblique images in separate test.
Final
Matrice 4T - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map.
Matrice 4E - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map.
Mavic 3E - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map..
Phantom 4 RTK - Nadir maximum speed (~10-12 mph) and oblique ~10-12 mph. A final map with one GCP on the nadir map.
Mavic 2 Enterprise Advanced - Nadir maximum speed and oblique maximum speed. A final map with one GCPon the nadir map.
Comparison in Agisoft Pro
I will compare the nadir maps by themselves in a chart.
I will compare the oblique maps by themselves in a chart.
I will compare all maps in a chart.
Matrice 4T will have 3 maps to compare RMSE
Matrice 4E will have 3 maps to compare RMSE
Mavic 3E will have 3 maps to compare RMSE
Phantom 4 RTK will have 2 maps to compare RMSE
Mavic 2 Enterprise Advanced will have 2 maps to compare RMSE
It will be interesting to see the winner from the various tests.
I have received the drone. Wow, what a great thermal solution. The thermal image, while being the same resolution as my Mavic 2 Enterprise Advanced, definitely has software enhancement and possibly is an upgraded sensor, especially with the Ultra/Super Resolution enabled.
We are still having winter weather, and it has been too windy to take the drone out for full flight testing.
I have already programmed the test flights on all of the drones for the test, but the weather will be a factor. I wanted to do it tomorrow, but wind gusts will be almost 30 mph.
The upcoming week does not have any promising days either for times that I am off from work.
Unfortunately, the best possible day I see in the near future is Wednesday, the 25th.
Here is how the test will be conducted, and I took people's ideas and implemented them the best I could.
Test name - DJI Small Form RTK Drone Shootout
Drones:
Matrice 4T
Matrice 4E
Mavic 3E
Phantom 4RTK
Mavic 2 Enterprise Advanced
Test Area
4.14 acres or 17,765 sq. meters
Mostly flat with elevation changes from 6 to 9 feet MSL.
Flight Altitude and GSD
I am not going to base the test on GSD as that would have 2 of the drones fly way too low. I will have all drones fly at 152 feet AGL. Two of the drones will have a greater GSD because of this, but it will still be under 2cm.
Flight Speed
The Phantom 4 RTK and Mavic 2 Enterprise Advanced will fly slower than the other drones since they cannot take images as fast as the newer solutions. The Mavic 3E and both Matrices will be limited to 15mph. I also will fly these same 3 drones at their maximum speed in separate flights as part of the test.
Nadir and Oblique
All drones will fly a nadir flight path with Altitude Optimization enabled and then fly another mission with oblique double grid. Nadir and oblique imaged flights will be processed by themselves and be part of the comparison test.
Overlap
Nadir - 80/80
Oblique - 75/75
Control Points
The first tests will have NO GCP's and use 5 check points. A final test will use a single GCP to see how much that helps in increasing absolute accuracy. All control points will be collected with an Emlid RS2+ or RS3 that will be receiving corrections from my RTK subscription service.
Drone RTK
All drones will receive corrections from my RTK subscription service. The base unit is located 19km away. I will pause any mission where FIX is lost and will only continue with FIX. If this is not possible I may use Red Toolbox to PPK all data.
Data Datums/Coordinate Systems
The Emlid GNSS rovers will collect in geographic Nad 83 (2011) with ellipsoid elevations and also in projection Nad83 (2011) New Jersey State plane US Survey feet, and orthometric elevations in NAVD88 US Survey feet using Geoid 18.
All drones will receive corrections in geographic Nad 83 (2011) with ellipsoid elevations since that this what the base is sending in.
All processing in all software will output in projection Nad 83 (2011), New Jersey State Plane US Survey feet with orthometric elevations in NAVD88 US Survey feet using Geoid 18.
Software
The main tests will be processed on Agisoft Metashape Pro. I will share the outputs on Agi Cloud or maybe even Arc Online.
On Agisoft, I will use the USGS workflow to remove questionable points on the sparse point cloud before final optimization.
Agisoft will output: 2D ortho map, 2D DEM, 3D Tiled Model
I will also process on Drone Deploy since it is a simple upload and walk away process. I will share the Drone Deploy page with all maps as layers.
In the future I will process in Pix4Dmapper, Pix4Dmatic and Mip Map.
I will use Mip Map's Gaussian Splatting for oblique images in separate test.
Final
Matrice 4T - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map.
Matrice 4E - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map.
Mavic 3E - 3 tests with just checkpoints: Nadir 15 mph, Nadir maximum speed and oblique 15 mph. A final map with one GCP on the nadir map..
Phantom 4 RTK - Nadir maximum speed (~10-12 mph) and oblique ~10-12 mph. A final map with one GCP on the nadir map.
Mavic 2 Enterprise Advanced - Nadir maximum speed and oblique maximum speed. A final map with one GCPon the nadir map.
Comparison in Agisoft Pro
I will compare the nadir maps by themselves in a chart.
I will compare the oblique maps by themselves in a chart.
I will compare all maps in a chart.
Matrice 4T will have 3 maps to compare RMSE
Matrice 4E will have 3 maps to compare RMSE
Mavic 3E will have 3 maps to compare RMSE
Phantom 4 RTK will have 2 maps to compare RMSE
Mavic 2 Enterprise Advanced will have 2 maps to compare RMSE
It will be interesting to see the winner from the various tests.
Goodness.. by the length of your message, I can see your happy with your new DJI toy 
.
Thank you for this. Well thought out, a lot of work and time consuming. Next to this, its a great example for people to prepare workflow and necessary documentation for operations. Thanks!
. Thank you for this. Well thought out, a lot of work and time consuming. Next to this, its a great example for people to prepare workflow and necessary documentation for operations. Thanks!
What a great opportunity to see all these tests, thank you so much!
Have you thought about making a video?
Later on, when you're more familiar with MT4, one thing I haven't been able to figure out yet is how to do 48px mapping. I've tried everything, but it still does them at 12px. I don't know if the same thing happens with MTe, any clue?
Have you thought about making a video?
Later on, when you're more familiar with MT4, one thing I haven't been able to figure out yet is how to do 48px mapping. I've tried everything, but it still does them at 12px. I don't know if the same thing happens with MTe, any clue?
Wow. I'm looking forward to this!
Of note. In the few tests that I ran with the P4R and M3E in Metashape. Ironically, the elevation was farther off using the diagonal obliques captured by the Altitude Optimization setting. It would be interesting if you duplicated a chunk and compared the results without them. It wouldn't require another flight.
