Seminal vesicle abscess is a rare urologic disease. Herein, we report our experience of the first case of a 41-year-old male patient with neurogenic bladder who underwent successful treatment of seminal vesicle abscess with rectal fistula after rectal decompression. Only a simple insertion of the rectal tube with intravenous antibiotics was able to remove the seminal vesicle abscess with rectal fistula without any percutaneous, transvesical, or transurethral drainage of the abscess. Rectal decompression should be considered in advance as a treatment of seminal vesicle abscess with rectal fistula before performing any invasive abscess drainage or fistulectomy.