Relatively safe, perfectly fine during the day, possibly not during the night, especially on weekends when there's drunk people about. I'd always walk around accompanied at night, and if you do there isn't much danger to worry about.

Colchester has its dodgy parts and gets a little unsafe at night as the city center has a small number of addicts who might bother you for a bit of change otherwise except for cat calling I have not experienced any sort of sexual assault or harrowing in the city