In Binary 320 is
00000000 00000000 00000001 01000000
-- 4th-- -- 3rd-- --2nd-- --1st--
There are two types of machine Little Endian and Big Endian. In Little Endian When any data is stored in memory the lower byte is stored in the lower address of the memory i.e 320 would be stored as -
01000000 00000001 00000000 00000000
1st 2nd 3rd 4th
Vice-Versa for Big Endian.
Now when you type casted the integer to the char the charcter pointer is accessing the value of the 1st byte which is obviously 64
Also need to mention that character pointer accesses only one byte of the memory whose address it is currently holding. If u need Further clarification ...Comment.