You need to understand the various formats you're using and what you want to convert to what before anyone can help you. There's no such thing as hexadecimal format. Hexadecimal is just a way of representing values. Packed decimal is a format. Binary is a format. Zoned decimal is a format. All of these can be represented in hexadecimal.
COBOL has its own set of formats that correspond to DFSORT formats. For example, COMP-3 corresponds to PD. COMP corresponds to BI or FI, etc.
See the following for more information on COBOL formats vs DFSORT formats: