Thanks for helping out with this. I was really trying a total different approach. Before implementing the logic that is there in my question, I had achieved this effect with a TOTAL DIFFERENT APPROACH and now I'm realizing that it was very dumb.
This was very smooth though, but was done using scrollTrigger's markers. This wouldn't work if turn the markers off. (very dumb)
See the Pen ZEPQKZL by abhiishek-10 (@abhiishek-10) on CodePen
But anyways, the solution you've provided is really awesome and perfect. Just tweaked a little and achieved what I wanted.
Here's the link to my solution. Hope this helps other people ; )
See the Pen abMdWgV by abhiishek-10 (@abhiishek-10) on CodePen
Abhishek Animates's post in How can add two ScrollTo instances on window? was marked as the answer