This is tangentially related to this slide, but I thought it might be really cool to add. I had always wondered how google maps generated the view on the right from satellite imagery, since I think even when you look at the left image with your eyes it can still be hard to distinguish what are roads, what are paths, and what are parks. I was wondering if Google had created some sort of machine learning program to process the image, but when I did some more research, I found that I was slightly off the mark. I had not considered that all the Google trucks (with cameras on them) could also give their position to Google, which gives those sections a much higher likelihood of being roads. Google is also able to track other data from people walking through areas, and what governments mandate to be parks, and so on. I just thought all this was interesting, because while the map on the right may seem to be simplified, it's actually an incredible feat of engineering and data/image processing.
aravmisra
@TheCrystalKeeper that's a cool thing to think about, and I actually think some companies (including Google) are doing work to amalgamate different imaging techniques to make the most accurate maps possible. For example, now when using Apple or Google maps while driving, sometimes at exits ramps you'll be shifted into more of a 3D-view that displays a more visually-accurate path. This mapping requires more than just the coordinate positions of roads, which is pretty fascinating. I also know this has huge applications towards AVs.
AlsonC
@aravmisra Super cool! I know companies like tesla have shifted their full self driving models from sensor based to image based, and I wonder if this ties into the breakthroughs in computer imaging!
SKwon1220
@TheCrystalKeeper This is really fascinating and I personally didn't know that Google was able to create maps with this level of detail through Google trucks and other agents. Given that Tesla, Waymo, and other autonomous vehicle companies are also having their self driving models to be image based, I wonder what the next steps are to further elevate the imaging process for mapping? Perhaps a more realistic and dynamic 3D rendering of surroundings could be incorporated on a map? Or maybe having an aerial view to introduce depth?
rcorona
@SKwon1220 to add to your comment here's a recent paper (https://arxiv.org/abs/2202.05263) from folks at Berkeley, Waymo, and Google where they use neural radiance fields to allow for the 3D rendering of blocks in a street. In my understanding, they train an individual NERF per some unit area and then swap between nerfs when rendering larger areas of a city.
JunoLee128
This is really interesting. Traditionally self-driving cars are thought to rely mainly on "vision" on the level of a driver, but it's interesting how new traffic patterns/strategies might emerge with satellite imagery in the mix too.
This is tangentially related to this slide, but I thought it might be really cool to add. I had always wondered how google maps generated the view on the right from satellite imagery, since I think even when you look at the left image with your eyes it can still be hard to distinguish what are roads, what are paths, and what are parks. I was wondering if Google had created some sort of machine learning program to process the image, but when I did some more research, I found that I was slightly off the mark. I had not considered that all the Google trucks (with cameras on them) could also give their position to Google, which gives those sections a much higher likelihood of being roads. Google is also able to track other data from people walking through areas, and what governments mandate to be parks, and so on. I just thought all this was interesting, because while the map on the right may seem to be simplified, it's actually an incredible feat of engineering and data/image processing.
@TheCrystalKeeper that's a cool thing to think about, and I actually think some companies (including Google) are doing work to amalgamate different imaging techniques to make the most accurate maps possible. For example, now when using Apple or Google maps while driving, sometimes at exits ramps you'll be shifted into more of a 3D-view that displays a more visually-accurate path. This mapping requires more than just the coordinate positions of roads, which is pretty fascinating. I also know this has huge applications towards AVs.
@aravmisra Super cool! I know companies like tesla have shifted their full self driving models from sensor based to image based, and I wonder if this ties into the breakthroughs in computer imaging!
@TheCrystalKeeper This is really fascinating and I personally didn't know that Google was able to create maps with this level of detail through Google trucks and other agents. Given that Tesla, Waymo, and other autonomous vehicle companies are also having their self driving models to be image based, I wonder what the next steps are to further elevate the imaging process for mapping? Perhaps a more realistic and dynamic 3D rendering of surroundings could be incorporated on a map? Or maybe having an aerial view to introduce depth?
@SKwon1220 to add to your comment here's a recent paper (https://arxiv.org/abs/2202.05263) from folks at Berkeley, Waymo, and Google where they use neural radiance fields to allow for the 3D rendering of blocks in a street. In my understanding, they train an individual NERF per some unit area and then swap between nerfs when rendering larger areas of a city.
This is really interesting. Traditionally self-driving cars are thought to rely mainly on "vision" on the level of a driver, but it's interesting how new traffic patterns/strategies might emerge with satellite imagery in the mix too.