We have to enable it using Admin Center.
Click Rights Management in the left panel
Once activated, go to the Library Settings
we can see new section called, "Information Rights Management"
We can set new "Policy Name" and "Description"
Once saved the RMS, when use opens the document it is assigned to RMS.
we can set the document library to "not to open in browser"
when user opens the document, it will be displayed the message.
though User download the document, it will be prompt the credentials.