I wasn't checking for color indices <16 so first 16 colors in palette were completely wrong and that was causing wrong indices to be calculated.