Eukaryotic chromosomes are folded into higher-order conformations to coordinate genome functions. In addition to long-range chromatin loops, recent chromosome conformation capture (3C)-based studies have indicated higher levels of chromatin structures including compartments and topologically associating domains (TADs), which may serve as units of genome organization and functions. However, the molecular machinery underlying these hierarchically three-dimensional (3D) chromatin architectures remains poorly understood. Via high-throughput assays, including in situ Hi-C, DamID, ChIP-seq, and RNA-seq, we investigated roles of the Heterogeneous Nuclear Ribonucleoprotein U (HNRNPU), a nuclear matrix (NM)-associated protein, in 3D genome organization. Upon the depletion of HNRNPU in mouse hepatocytes, the coverage of lamina-associated domains (LADs) in the genome increases from 53.1% to 68.6%, and a global condensation of chromatin was observed. Furthermore, disruption of HNRNPU leads to compartment switching on 7.5% of the genome, decreases TAD boundary strengths at borders between A (active) and B (inactive) compartments, and reduces chromatin loop intensities. Long-range chromatin interactions between and within compartments or TADs are also significantly remodeled upon HNRNPU depletion. Intriguingly, HNRNPU mainly associates with active chromatin, and 80% of HNRNPU peaks coincide with the binding of CTCF or RAD21. Collectively, we demonstrated that HNRNPU functions as a major factor maintaining 3D chromatin architecture, suggesting important roles of NM-associated proteins in genome organization.