As some people have said above, don't give up your hobby. It's important to have something else for you outside the daily routine. Just for your well-being. Maybe you'll have a period without much time to make music but you'll come back to it later.
I'm 38, one daughter 2 years old, awaiting a second child in 4 months, with a job where I'm away from home half of the year, making works in my house to enlarge it (that includes a new room for the baby also) during my vacation and spare time. I feel like my life is a race, I have the feeling to be running all the time, but under no circumstance I'm thinking about giving up music.
So far I'm doing something similar to J0n35y, I try to keep one evening/week dedicated to making music when I'm at home. I consider it like practicing a sport.
It's all about finding a balance in your life, if you sacrifice everything to something you'll end up frustrated and your children will feel it.
Think also your kids might like to see their dad making music when they grow up and you can have some fun moments with them around that, even if your wife don't enjoy music particularly.
I had a really cool moment with my daughter last week, we went to my brother's for dinner and she wanted to play drums. I played the kick and hats with the feet and she played the snare, toms and ride with the sticks on my knees. We had a blast! Now she wants to see all the time the video we did! It's also something I will be very happy to show her when she is older.
Being a parent is the best thing in life and my love of music (making and listening) is something I would like to transmit to my children.