Unexplained swelling over left eye

I’m 29F, take 2000 IU vitamin D3 daily, non-smoker, have about one drink a month, no drug use. 

On NYE a couple weeks ago, I was playing a dice game at a party when I suddenly felt a strange sensation around my left eye. When I looked in the mirror, I saw that the tissue underneath my brow bone had swollen down over my left eyelid. I figured it would go away by morning. However, when I woke up on January 1st, the swelling had become even worse (see pic in 1st comment). 

Over the course of the day, the swelling receded. When I woke up on January 2nd, it was back. It receded again over the course of the day. When I woke up the morning of January 3rd, it was back. After that, it went away for a couple weeks, then reappeared again a couple days ago when I woke up. It has not come back again since then. 

I saw my PCP, who thought it was allergy-related and said to see an allergist. I saw an allergist, who thought it wasn’t allergy-related (more on that below), and said to see an ophthalmologist. I saw an ophthalmologist, who said it was allergy-related. So, there’s a lack of agreement among the docs as to the cause. 

Here are the reasons that the allergist (and I) don’t think it’s allergy-related: I’ve had contact allergic reactions in my eyes before, but they always follow the same pattern. I would touch a pet, then touch my eye, and immediately my eye would become itchy and water, and my lower eyelid would swell slightly. The reaction would respond to antihistamines and never lasted more than about 4 hours total anyway. The NYE swelling, on the other hand, happened when I had not just touched my eye, and it did not affect my eye itself at all, just the tissue above it. It did not respond to antihistamines and persisted for multiple days, seemingly getting worse at night. 

Does anyone have any idea what could have caused this? I had a basic blood workup done when I saw my PCP and everything was normal, including thyroid. Allergist ordered C1/4 bloodwork to check for hereditary or acquired angioedema, that came back normal. Had rheumatological bloodwork about a year and a half ago, positive ANA but titer of 1:80 and no specific autoimmune diseases popped. Pressure in both eyes was normal. Ophthalmologist said it could have been an internal stye, but he’s never seen one come and go so quickly. 

Any ideas? Thanks in advance.