vcubingx is a good youtuber that made a video on algorithms like this, I'd recommend checking out his channel to get a better understanding on these types of algorithms :)
colinsteidtmann
Wow, I'm surprised ML is fast enough
pranavkolluri
Is there a particular advantage to using ml as opposed to a more conventional technique (that presumably also has dedicated hardware on the camera sensor or the image processor to take care of this?)
rcorona
@pranavkolluri, demosaicking is not my area but coming from a computer vision perspective machine learning techniques can generally provide more flexibility in terms of the learned solution or rule that an algorithm uses to aggregate features (e.g. such as aggregating pixel information).
For example, one way of averaging features in computer vision is through max or mean pooling, both of which are very rigid and straight-forward methods for aggregation. In contrast, machine learning methods like transformers learn their own aggregation methods which have different behavior in different contexts (e.g. in some images it might be better to mean-pool, in others to take the min or max, etc.), allowing for more flexibility.
Going back to this slide, my guess would be that using machine learning allows for more adaptability to a wider variety of conditions than traditional hand-engineered methods.
OnceLim
I'm wondering if ML is necessary to compute demosaicking when it is not computationally intense to calculate.
RishSharma7
To respond to Lim, I don't think ML is "necessary" in this specific situation, but excessive expense and effort in ML generally come from computationally intense calculations, so I would assume it's not a bad idea to use ML there either. But you're right, I'm sure its usefulness goes up exponentially as computations become increasingly more intense.
zeddybot
I wonder how large of an area the ML model considers when performing demosaicking. If it just considers the neighboring pixels, it seems hard to believe that it performs that much better than traditional algorithms. But the larger the surrounding area that the model receives as context, the trickier such a model is to train.
vcubingx is a good youtuber that made a video on algorithms like this, I'd recommend checking out his channel to get a better understanding on these types of algorithms :)
Wow, I'm surprised ML is fast enough
Is there a particular advantage to using ml as opposed to a more conventional technique (that presumably also has dedicated hardware on the camera sensor or the image processor to take care of this?)
@pranavkolluri, demosaicking is not my area but coming from a computer vision perspective machine learning techniques can generally provide more flexibility in terms of the learned solution or rule that an algorithm uses to aggregate features (e.g. such as aggregating pixel information).
For example, one way of averaging features in computer vision is through max or mean pooling, both of which are very rigid and straight-forward methods for aggregation. In contrast, machine learning methods like transformers learn their own aggregation methods which have different behavior in different contexts (e.g. in some images it might be better to mean-pool, in others to take the min or max, etc.), allowing for more flexibility.
Going back to this slide, my guess would be that using machine learning allows for more adaptability to a wider variety of conditions than traditional hand-engineered methods.
I'm wondering if ML is necessary to compute demosaicking when it is not computationally intense to calculate.
To respond to Lim, I don't think ML is "necessary" in this specific situation, but excessive expense and effort in ML generally come from computationally intense calculations, so I would assume it's not a bad idea to use ML there either. But you're right, I'm sure its usefulness goes up exponentially as computations become increasingly more intense.
I wonder how large of an area the ML model considers when performing demosaicking. If it just considers the neighboring pixels, it seems hard to believe that it performs that much better than traditional algorithms. But the larger the surrounding area that the model receives as context, the trickier such a model is to train.