What is the difference between the PatchMatch algorithm demonstrated here and Generative Fill used in Adobe Photoshop?
weinatalie
I looked into the PatchMatch algorithm and found that it works by splitting the image into several, same-sized patches. For each patch that lies within the bounding box of the missing region, the algorithm determines the nearest neighbor patch (i.e. the most similar patch) outside of the bounding box, and uses this information to estimate the values of the missing pixels. We can improve the results of the algorithm in various ways—for example, using adaptively-sized patches or k-nearest neighbors.
davidmyang
To complete a response to adam2451, Photoshop's generative fill is based on GenAI. Thus, in addition to using the original photo itself, Generative Fill has a huge dataset of images that the model is trained on for it to seamlessly add to the original image.
JunoLee128
Does the PatchMatch algorithm not need to learn about other similar pictures first? (Or are certain parameters essentially trained through data?) I'm interested in the differences between data and non-data driven approaches
brianqch
Looking into this a bit further, it seems like the paper on PatternMatch algorithm talks about the possibility of using this for videos as well. This makes sense because pattern matching could then be used on the frames as they change.
GH-JamesD
Interesting to see how diffusion models have evolved from these initial pattern match image completions. The semantically guided adobe photoshop fill is a pretty impressive example.
What is the difference between the PatchMatch algorithm demonstrated here and Generative Fill used in Adobe Photoshop?
I looked into the PatchMatch algorithm and found that it works by splitting the image into several, same-sized patches. For each patch that lies within the bounding box of the missing region, the algorithm determines the nearest neighbor patch (i.e. the most similar patch) outside of the bounding box, and uses this information to estimate the values of the missing pixels. We can improve the results of the algorithm in various ways—for example, using adaptively-sized patches or k-nearest neighbors.
To complete a response to adam2451, Photoshop's generative fill is based on GenAI. Thus, in addition to using the original photo itself, Generative Fill has a huge dataset of images that the model is trained on for it to seamlessly add to the original image.
Does the PatchMatch algorithm not need to learn about other similar pictures first? (Or are certain parameters essentially trained through data?) I'm interested in the differences between data and non-data driven approaches
Looking into this a bit further, it seems like the paper on PatternMatch algorithm talks about the possibility of using this for videos as well. This makes sense because pattern matching could then be used on the frames as they change.
Interesting to see how diffusion models have evolved from these initial pattern match image completions. The semantically guided adobe photoshop fill is a pretty impressive example.