No. Do not use hot water.
The standard Brita filter is rated for cold tap water only. Maximum input water temperature is 85°F (29°C). Minimum is 32°F (0°C). Hot water can damage the filter media and void its certification.
Filter cold tap water as normal, then heat the filtered water separately for your tea.
Using hot water is product misuse. The filter change indicator logic does not account for this, so it won't adjust the filter life reminder accordingly.