This shield is also able to charge your battery.
When you're new, don't start with a separate ESP (ESP-01 to ESP-14) module.
Get yourself a proper development board: NodeMCU - WeMos D1 - WeMos (UNO form factor) - Witty Board
ESP power requirements are 3.3V±10%@350mA
The ESP8266 IS NOT 5V tolerant on ANY pin (see item 5.14 on page 25 of the FAQ)
Your battery has a nominal voltage of 3.7V, but it varies from 3.0V to 4.2V depending on whether it is empty or fully charged. So the 4.2V will probably kill your ESP is you connect it directly.
A method I have used before is boosting the battery voltage to around 5V using a step up converter module.
And then a 3.3v linear regulator to make a nice and steady 3.3v (for example LD1117v33) with plenty of power. Maybe not the most efficient way, but it works fine.