You are not alone on this one. I chalk Avengers down on my list of things that everyone likes but I don't so much, which on here is along with Lego Movie and Kingsman. I don't dislike it, but I find a lot of it extremely boring and I say that as a huge fan of Whedon (Buffy is probably my favourite tv show ever).
I also went to see Avengers with a group of friends, the most of us who hadn't been to the cinema together in years.
I fell asleep for a good portion of the movie, as did my friend next to me.
I will caveat that by admitting that both of us had sunk about 8 or 9 pints that day.
I watched it again when it come out on DVD, with my wife and I had the same problem. My wife was bored out her face and I just totally switched off with all the technobabble, which I thought plagued the movie. The teaseract stuff i find beyond mind numbing and and I found it far too long.
I fell asleep again watching it, this time no alcohol involved.
There was a lot to like in it though, some really memeorable moments. It's a 6/10 movie in my head, that would have been 8 or 9 if they'd cut out all the sleep inducing moments.
It's nowhere near as poor as Age of Ultron though, which both my friend and I left the cinema actually angry about, because it was so needlessly long and tiresome.
None of this is retroactive for me, I think I posted the same shit at the time it came out.