By Iviwe Booi
Have you ever wondered what your skin looks like beyond the colours visible in a selfie? An ordinary camera records light using only three broad channels: red, green and blue. A hyperspectral camera separates the same light into many narrow wavelength bands. This richer view can reveal subtle differences that an ordinary photograph compresses or misses. The challenge is that hyperspectral cameras are specialised, expensive and not widely available.
Zhao and colleagues asked whether artificial intelligence could reconstruct this missing spectral information from a normal RGB image. Their model, called HSGAN, uses two neural networks that learn through competition. A generator creates a hyperspectral estimate, while a discriminator checks whether that estimate resembles real hyperspectral data. It is similar to a student improving an answer each time a strict marker points out what still looks unrealistic.
Figure 1: A simplified overview of RGB-to-hyperspectral reconstruction. HSGAN uses the red, green and blue information in an ordinary photograph to estimate an image containing many wavelength bands. Adapted from Zhao et al. (2024).
The researchers first trained the generator to learn the general relationship between RGB colours and hyperspectral information. They then added adversarial training, allowing the discriminator to push the generator towards more realistic outputs. The model also combined information at different image scales and used attention mechanisms to focus on useful spatial and spectral patterns.
When HSGAN was tested on five established hyperspectral datasets, it produced more accurate reconstructions than the comparison methods reported in the study. It also remained more consistent when realistic noise was added to the RGB images. This is important because photographs captured outside a laboratory are rarely perfectly clean.
However, reconstructing outdoor scenes is not the same as reconstructing human skin. The Hyper-Skin dataset was developed to support this next challenge by providing paired facial RGB and hyperspectral images across visible and near-infrared wavelengths. My honours project uses these data to compare two strategies: first training HSGAN on the larger NTIRE dataset and then fine-tuning it on Hyper-Skin, or training the model directly on Hyper-Skin.
This does not mean that a selfie can already diagnose a skin condition. The reconstructed spectrum is still an AI-generated estimate and must be carefully validated. However, if these methods become accurate and reliable, they could make spectral skin research more accessible and reduce dependence on specialised imaging equipment. The bigger story is not about teaching
a camera to see through skin, but about using AI to recover information that a normal camera leaves behind.
References
Zhao, Y., Po, L.-M., Lin, T., Yan, Q., Liu, W. and Xian, P. (2024). HSGAN: Hyperspectral Reconstruction from RGB Images with Generative Adversarial Network. IEEE Transactions on Neural Networks and Learning Systems, 35(12), 17137–17150.
Ng, P.C., Chi, Z., Verdie, Y., Lu, J. and Plataniotis, K.N. (2023). Hyper-Skin: A Hyperspectral Dataset for Reconstructing Facial Skin-Spectra from RGB Images. Advances in Neural Information Processing Systems, 36.
Leave a Reply