My small business is growing pretty fast. Keeping up with the invoices, bills, and bank statement is driving me nuts! And don't ask about tax time...
When completing a task starts to drive you nuts - that might be a good time to look to outsource it!
But, if your finances afford you the ability to hire outside help and your company is growing fast, now sounds like a good time to hire extra help. It will allow you the ability to stay focused on the important things like growing your business, instead of dealing with the daily headaches of keeping up with the paperwork.
Of course, you will want to make sure you have sufficient capital in your business before looking to expand. You don't want to expand too quick and then suffer a downturn in business and not have the capital to absorb the ups and downs.
Once you face the choice in your business of either doing what your core business is or doing bookkeeping, I think you're nuts not to consider a contract bookkeeping service.
Of course, starting off your business by doing your own bookkeeping is a good learning experience, but after a while, it just interferes with your core business.
And as for tax time, most bookkeepers are not tax experts - find an Enrolled Agent or a CPA that does taxes.