I violated the $25 rule twice. Once was for a Zonophone Type A at $27. The other was for a Columbia BO Graphophone with a half-barrel cabinet full of cylinders. That cost $40 so I rationalized it as $25 for the phonograph and $15 for the cabinet and records.
But we had no clue what was rare and what wasn't. I paid $25 for a Gem B because I had never seen one, or even heard of one, so I assumed it was rare. Duh. But my Graphophone Type A with gutta percha reproducer and morning glory horn with stand was $17.
The first phonograph I ever bought was a VV-IX for $5. The second was a Columbia BKT for $12. The third was expensive: an AB Graphophone with 4 Edison Concert cylinders for $20. I still have that one.
Cylinders were 10c, tops. Discs were much less.
However, as shocking as those prices seem today we have to bear in mind that my father earned $12,000 a year, and my mother didn't work. Not to mention they put two of us through college. So $25 was a big investment. Fortunately it paid off: I sold the majority of my collection when I moved out after college and we made a very good return on the investment.
EDIT to add: the photograph in question was for a newspaper article, as I posted about earlier. That article, and a later short local TV appearance, led to a lot of great finds. I bought a mint Zonophone Parlor for $7 after the article came out. One night we got a call and someone brought over a Graphophone Type Q in near-mint, complete condition. As with the Gem, I hadn't seen one and assumed it was rare. The owner proceeded to say that he found it in a dumpster, and an antique dealer offered him $10. My father thought about it a bit and finally asked "would you take $10 for it?". The guy replied "sure, from him (pointing to me), but not from a dealer!" Being a 'cute' kid paid benefits.