Appendix p. 69 of the manual lists thumbnail support. AVI is not one of the video formats which supports thumbs. The only luck I have had is importing mpeg4s into iTunes and adding cover art there. This will show on the WD TV HD. Hopefully they will come up with a solution for AVIs and MKVs.
In other players I've been able to just put the .avi/.mkv into a folder, and then put folder.jpg in there with it and then see the higher level folders with the cover art. It seems this doesn't work for WDTV. Anyone figure out a way around this yet?