1. Have you ever worked with an agency before? YES
2. At what point in your startup's development did you decide to outsource? At the very beginning. In the beginning it doesn't make sense to hire a marketing staff, and even one marketer that can be broad and deep is expensive. The decision to bring in-house, to me, a financial one. Agency margins are brutal, so when the company is able to allocate resources to marketing staff to bring costs down, it makes sense. Then I continue to use agencies for specific "scalpel-like" expertise that is not an ongoing need (e.g. brand definition or re-brand).
3. How did you go about selecting a vendor? Word of mouth if possible. I have also used Google for specific combinations of expertise, or even working backward from companies I admire.
4. When do you never use an agency? When you're not getting the ROI from the costs. Sadly, this is very often the case because agency incentives are inherently misaligned with the client's.