Why can't I speak Portuguese?

I don't know my fathers Native language, but is it my fault?
My father immigrated to this gnarly country in the last 60's from Brazil. He met  my Midwestern mother in The great New York City. My older brother was born in 75' and I followed six years later. We were both born New Yorker's. As I grew up my external family came to visit from Ohio and Brazil. Ohio was easier and cheaper to visit than Brazil, so I went there more often. I have been to Brazil 3 times in my life. 
Growing up in my internal family I have to say I herd mostly english. My father and aunt who lived with us at the time, spoke portuguese back to each other. I don't remember siting down for any serious lessons, or for that matter running way from any.
Now I'm thirty and during dinner tonight the topic came up of why I don't speak Portuguese. My father claims it's because I never wanted to learn. I say bullshit. Perhaps it should have been taught to me when I was a child and my memory banks were fully open for business to absorb. I think it would have been a lot more intuitive than learning now. So at thirty I have decided it is my responsibility to learn this language. And I totally will.
But yet I ask my original question. Honestly, who's fault is it I don't know my fathers native language?