Recall that [-y, x] is perpendicular to [x, y]. This is doing such a projection onto the perpendicular of PQ with origin P

Adding onto RCD's comment, the reason why $(-y, x)$ is perpendicular to $(x, y)$ is because the dot product is 0.

Let $\vec n$ be the normal vector to $\vec{PQ}$ and $\vec x$ be the vector $(x, y)$ relative to $P$. Then, the shown equation is essentially the dot product of x and n, $\vec x \cdot n$.

If you were to draw line from P to $(x, y)$, then you end up with a triangle where the bracket is essentially $||\vec x|| cos \theta$ where $\theta$ is the angle at point P.

So the dot product is proportional to the distance from line PQ since it is $||\vec x|| ||\vec n|| cos \theta$.

oof my latex doesn't render as pretty as I would like

A quick note for the project implementation: ABC may be either clockwise or counterclockwise. As a result, you need to check for it or the equation here may give the inverse of the correct value.