Yes, starting your business part-time while you hold down a full-time job is difficult, but that's how I started initially.
I think being upfront and honest with your clients is the best policy.
I would suggest that you deal with clients as best you can, after hours ~ evening meetings and working weekends to complete projects.
I recall handling three clients this way before deciding to take the plunge full-time. I would meet with them in their offices after hours and/or over dinner, and work on their jobs in my spare time. There will also be times when you need to contact or meet clients or third parties during business hours. All I can suggest here is that you take time off when you need it and/or work through your lunch hours.
I know it's difficult, but it's really all you can do if you wish to put that toe in the water first before committing yourself full-time to your business.
During this initial stage, it's important to tell your clients your situation ~ they will understand and they will respect you for your integrity.