It's probably more to do with the container file format than the video codec H.264. The DLNA server generally streams the encoded video to the client without touching it (unless it is re-encoding on the fly, which AllShare doesn't do), but it can be very dependent on the container file format. Sometimes the video creation software will change the container file format internally when you cross a certain size threshold, leading to the puzzling symptom of a file format that seems to work sometimes and not others. You didn't say what the container file format is (e.g., mp4, avi, mov, mkv, etc.), so I can't offer any specifics.
But in any case, you might want to try Serviio as a server. It generally does a good job with the Samsung Blu-Ray client, and it will re-encode on the fly if necessary, although you would need a very fast computer for that to work with HD video. Samsung's AllShare software is not one of the better DLNA servers.