I enjoy how this video visualizes how CPU versus GPU works
CPU can only do one thing at a time (technically, per thread), while GPUs can do a lot more things at a time
This is very good because each pixel is just some combination of simple calculations done over and over again to determine the final value for that pixel (especially Normalizing and then the Z-value depth info)
This idea is especially the exact same concept for machine learning where there's a lot of dimensions and a whole lot of simple calculations
There is interest in using the "embarrassingly" parallel nature of the GPU in the computational sciences and simulations. One interesting thing you can do yourself is write shaders for cellular automata. You can treat a cell as a pixel, for example, in Conway's game of life, encoding the state in the RGB values and getting the state of the neighbors by sampling the previous game state. This gets you incredibly fast simulations.