Sussex Police spokesman said shortly before the crash, a police vehicle had attempted to stop the Mazda. Two other men in the car were injured.
The incident has been referred to the Independent Police Complaints Commission.
'Early stages' A force spokeswoman said the stolen car sped off when it passed a stationary police car in Mill Lane.
The police car then followed the Mazda for a few seconds before it collided with railings.
The man, believed to be from Liverpool, was confirmed dead at the scene.
The two other men were taken to the Royal Sussex County Hospital with non-life-threatening injuries.
One of the men remains in hospital and the other has since been arrested on suspicion of death by dangerous driving and aggravated vehicle-taking.
Ch Insp Natalie Moloney, of Sussex Police, said: "We are in the early stages of the investigation to try and establish the exact circumstances that led up to this fatal collision."