Well, your problem is with the coordinates.
I think you must only have
Also, try to use u16 instead of short, short is signed and is bad for SNES (your u8 for mapcol is correct).
I think you must only have
CitationX does not need to be multiply by height
//512 pixel width 512/8 = 64
u16 *ptrMap = (u16 *) &mapCol + (OBJY>>3)*64 + (OBJX>>3);
Also, try to use u16 instead of short, short is signed and is bad for SNES (your u8 for mapcol is correct).