Marie,
First off, you doctor should be beaten to a pulp for that stupid statement. I would suggest that you find another. He obviously doesn't understand diabetes or you. I am amazed at what is said by some 'professionals'. Rather than offering suggestions, encouragement or referring to someone qualified to do so, they leave a patient without hope.
A brief, off-topic story: I grew up with pain. I barely recall a single day of my life without it. It began when I was very small. I have since learned that it was most likely Juvenile Rheumatoid Arthritis. At age 11 I was diagnosed with Charcot Marie Tooth, a neuromuscular disorder. At the time, the doctors said there was nothing they could do for me, but they wanted to monitor my condition. For the next 10 years, I was subjected to embarrassing, unpleasant visits to an orthopedic clinic where they would parade me in front of a group of interns and students. Sometimes there were as many as 15 in the room, gawking at me, nodding sadly, whispering and even pointing.
The best 'advice' they could give me was 'don't do any physical activity'. I didn't listen. I played sports, I lifted weights, I bicycled and did anything that other boys my age did. I didn't have an easy time doing some of it and I would hurt before, during and after, but
I didn't quit. Had I quit, I don't believe I would be here today.
Back to topic: I exercise alone. I always have. I don't require much outside encouragement to keep at it. I can see how having a partner works well for many.
In addition to your walks, do you do any weight training? A book I read recently referred to burning fat this way: Consider how the Mafia handles a problem. When they have a problem, they call up some 'muscle' to take care of it. For weight loss or fat loss, we can do the same thing. We can't call in two or three guys to 'take care of it', but we can use weights to put more of our muscles to work on it.

I believe this idea can be helpful with improving diabetes control.
I hope some of this rambling helps.
Travis