Well, since they can remain silent, here's a solution (although the problem seems to be about logic, and the solution is too simple):
red color - 1 second of silence
blue color - 2 seconds
green - 3 seconds
The 100th person remains silent for 1/2/3 seconds (depending on the color of the next wise man's hat) and names the color randomly. The next wise man looks at the color of the person in front, remains silent for the necessary time, and names his color. and so on.
Because there is a possibility that someone might miscount the seconds...
Example:
I had a friend who thought ten seconds had passed when only three had! It's a real case.
The solution here should be built a little differently... and strictly mathematically...
