What you are looking for is a voltage supervisor. (Note: I'm not recommending you buy them from that source, just providing an example of what you need to look for.)
The ones under that link are three pin devices that connect to your battery and ground, have an output that goes low when the battery voltage drops below a preset level. There are various models that have different cutoff voltages.
Since you mention price, the linked units are around $0.29 each if you buy 10 of them, which fits well with your price target.
Since you want non-SMD, I will mention that they are available in SMD and TO-92 (non-SMD) housings.
There also adjustable ones from various manufacturers.
There are also voltage detections circuits that you can build from discrete parts (example,) but making them work reliably might involve more complications than you want to deal with.