A great article on why Prometheus is so RAM-hungry -- which lends itself to being guidance on what can be done about it.

High memory usage - Prometheus is consuming excessive memory due to high cardinality metrics or large retention settings.