The "select new" button has some unfortunate edge cases where it is awkward. Sadly you've run into one of them.
The "select new" button only selects new photos for this particular launch of the application. Any photos the application has already seen are no longer marked as "new". So if you want to avoid this behavior, you can simply launch the app twice, making sure not to press "select new" the first launch.
Also note that the "select new" button is a toggle, it will also "deselect new" photos, but with the same property in which only photos that were new at the time of this particular launch will be deselected.
I would love to hear your suggestions for a better "select new" behavior.