The man, who was in his 70s, fell as he left Manhattan's Jacob Javits Convention Center just 25 minutes after receiving his shot, New York State Health Commissioner Dr. Howard Zucker said in a statement.
He was pronounced dead at a local hospital shortly after he was rushed there.
It's currently unclear if the man's death is linked to the vaccine, but an investigation is now underway.
'Initial indications are that the man did not have any allergic reaction to the vaccine,' Zucker said in his statement.
The man's sudden collapsing followed a mandatory 15-minute observation period, in which he 'exhibited no adverse reactions or any distress,' Zucker added.
The FDNY said the patient was taken to the hospital by Senior Care just before 11 am Sunday, where he died shortly after arriving.
The identity of the man has not yet been revealed by officials. It's also unclear which type of COVID-19 vaccine he received.
In his statement, Zucker stressed that he and other public health experts believe the vaccine 'is safe, and together with continued vigilance including wearing a mask and social distancing, it will bring an end to this pandemic.'
'I encourage all eligible New Yorkers to get vaccinated,' Zucker continued.
So far, nearly one million people in New York City have been vaccinated, with 1.5 million residents having received their first doses statewide.
In total, nearly 42 million doses have so far been administered in the US.
One dose of a vaccine has been administered to 31,579,100 million people, which equals to 9.6 percent of the population. While over 9,147,000 Americans have now received both doses of a vaccine, amounting to 2.78 percent of the population.