First, I love this question and the opportunity to address it in this happy space. It's sticky. A surface answer might be: Pinterest! Tumblr! Instagram! Magazines! Design books! And those resources have their place, certainly. In fact, I go to them on a nearly daily basis.
But if you really want to get into the nitty gritty about "inspiration hunting," things suddenly get a little more serious, as this is the constant subject of debate in the design world. After all, how does one draw the line between "inspiration" and "emulation" or "imitation" or "homage"? What is original anymore, or rather — is originality even important? I'd like to think so. Life would be pretty boring otherwise. (Unless you're Andy Warhol, and then you totally got away with it. But he was an original in being an unoriginal… so yeah.)
To be clear, I think imitation has its place. "Repeat after me" is a viable method of teaching and learning. It's a survivalist method of exploration — practice through the process of versioning. Likewise, art education teaches us to copy.
On a professional level, imitation gets tricky (as it should) and raises some debate over ownership of ideas. If you are looking at what someone else is doing and decide to adapt their existing work for your own purposes or your own content, then is the final result really yours, or does it belong to the person whose ideas inspired you? Do you credit that person as "inspiration" or are you just flat-out copying? I'm not sure. But this conflict can make the blogosphere a not so fun environment sometimes because (let's be real) copying happens all the freaking time. Peronally, it makes my heart sink to see it happen. (We are a much smaller community than you think. We notice.)
So, here's my advice on inspiration hunting. Tread lightly, and think critically. Pretty things are easy to find and duplicate. Key word: easy, which usually means the end product isn't all that inspiring. When you're hunting, don't just look for pretty things. It usually leads to unoriginal (but admittedly functional) work — the "chic banal" as Zac Posen says on Project Runway...
No, pretty is not enough. Look behind the pretty. Ask yourself why or how did that person come up with that idea? Why does that work? Break it down. Dissect the ideas. It will help you not to piggyback on someone else's ideas, but to instead appreciate their process as a manner of inspiring your own. Rinse and repeat. The practice of making observations and asking questions has the power to help you uncover the opportunities and needs that can lead to great design work.
When they say curiosity goes hand-in-hand with creativity, this is exactly why. For a creative person, inspiration-hunting is not so much an activity as it is a learned manner of thinking about everything around at all times. This helps to find space for something new — wiggle room in the universe, if you will. When I'm designing, I look for a need, and try to find interesting ways to address that need through what I'm designing, be it a blog post or a column idea, a logo or a website for someone else. For example, every single piece of Note to Self content, from this very Advice column to Words for the Week, to The Print Shop to Hers/Mine was thought of with you and our needs as consumers of information in mind. Need is inspiring.
Was that too much? I hope not. After all, design is really problem solving. It's meant to be sticky and challenging and mind-bending... No one is perfect and imitative mistakes happen, but as long as we recognize that it's part of the journey of learning a skill or stretching a talent, I think that's the most important thing. Comments section, let 'er rip!