Aim: The purpose of this IRB-approved, retrospective study was to compare image quality between 2D and high-resolution 3D, T2-weighted (T2WI) magnetic resonance imaging (MRI) sequences and to investigate the additional value of ultra-high b-value diffusion-weighted imaging (DWI; b=2,000 mm/s 2 ) for both rectal cancer staging and evaluating treatment response. Materials and Methods: From 12 February to 24 August 2016, 26 consecutive patients (22 males, four females; mean age: 61.9±14.0 years) with histologically-proven rectal cancer. In total 31 examinations [12 prior to and 19 after chemoradiation (CRT)] were included. The patients underwent pelvic MRI on a 3.0-T scanner (Magnetom Skyra, Erlangen, Germany). Three radiologists (3, 4, and 5 years of experience in MRI, respectively) independently assessed all images and rated the image quality of DWI (b=800 mm/s 2 ), apparent diffusion coefficient map, DWI (b=2,000 mm/s 2 ), 3D sagittal T2WI, 3D axial T2WI, 2D sagittal T2WI, and 2D axial T2WI of each patient, respectively. In addition, signal intensity ratios (SIR) were calculated between rectal cancer and obturator internus muscle (background) in all patients after CRT on DWI (b=2,000 mm/s 2 ) and correlated with histopathological regression grade (RG). Results: Tumor delineation was significantly better by 2D T2WI than 3D T2WI both before and after CRT (before CRT: Z=–3.2, p=0.02; after CRT: Z=–4.408, p〈0.001; all: Z=–5.192; p〈0.001) and was the preferred method, although image quality ratings were not significantly different (3D sagittal: 4.00±0.48; 2D sagittal: 4.03±0.34, p=0.713; 3D axial: 3.85±0.61, 2D axial: 3.78±0.64, p=0.537). Independent t-test showed significantly higher SIR between those with RG 1 or 2 (moderate response: mean score=2.02) and those with RG 3+4 (good response: mean score=0.8) (t=3.044, p=0.011). In those with RG 4 (complete response), SIR of b2000 was 0.946 compared to a 1.41 average of the whole cohort. In two patients, tumor was invisible on b2000 following CRT (RG 3 and 4, respectively). Interobserver agreement was mostly good (≥0.6) regarding image quality assessment, except for poor agreement (=0.4) in DWI (b2000) between the two less-experienced readers. Conclusion: In conclusion, 3D T2WI might be useful for evaluating response to neoadjuvant therapy in a comprehensive, cost-effective protocol, where 2D imaging seems to be preferable. In addition, DWI (b2000) may be beneficial in assessing both the primary and the residual tumor after CRT in rectal cancer and SIR may be helpful in assessing response to CRT.