For starters, the vast majority of apps, of the million out there, do not sell well, whether open or closed source. So don't expect any app to sell well unless it is a stand-out app and well marketed.
For iOS apps, someone would need to have paid Apple $99/annum for an iOS Developer program enrollment, plus have a Mac, in order to be able to download your open source, build the app, and load it on their device, without buying your app. Not many end users would do this, even if they had the expertise.
However, there is little or nothing to prevent some other developers from doing this, even submitting it to an app store under their own name, perhaps undercutting your price, or adding advertisements to a free app.
There are more and more developers doing the latter. Some reportedly even using content that is copyrighted and not open source. So competition is almost guaranteed if your app gains any visibility. Your advantage over the copycats, if any, will be brand name recognition from marketing, PR, or fame, if you have any, as many of the app stores do attempt to protect registered trademarks.
ADDED:
If you distribute your open source under the GPL v3 license, there may be some legal opinion that using code so licensed is incompatible with Apple's iOS App store DRM. But you, as the 100% copyright holder, could submit your own app to the iOS App store yourself, since the license doesn't apply to the owner. However, you may be able to consult an attorney about whether you would have grounds to sue anyone else who took your code under GPL v3 license, and tried to do likewise. This might cost you money to enforce though. But IANAL, so consult a lawyer if you want to think about using this strategy.
ADDED later:
As of Xcode 7.x, one no longer needs to pay Apple a $99/annum developer enrollment fee to install apps from a Mac to their own connected iOS devices. Only a valid Apple Developer ID and knowledge in how to use Xcode is needed.