Replace generic icon of image files to the actual image
Xentient Thumbnails is an amazing, free Windows program, belonging to the category Software utilities with subcategory Task Bar Start Menu & Explorer (more specifically Explorer & Shell Enhancements).