One thing that I have thought of for "k_d" is that there are constantly defined value for each object, like in real life as written in textbooks and papers like this: https://arxiv.org/abs/physics/0612101
Are these constant the exact same or similar idea for objects in the digital world?
Unfortunately, I can't comment too much on that paper. However, these shading equations are physically-inspired and the k_d constant in the case of diffuse shading just represents how diffuse the object is.
Are there situations where we would want to know the dot product, or at least magnitude of the dot product even if the angle is such that n*l is negative? For example, what if our surface is a thin sheet of paper and we want to get that golden brown glow through it with a light from behind?
still a bit confused about why its I/r^2?
I think it's because we are looking at the surface area. In a sphere, the surface area is 4pir^2. As you go farther from the center, the intensity decreases so intensity is inversely proportional to the surface area. Therefore it's I/r^2.