When does Taraaweeh prayer start in Ramadaan – on the first night or the second?

It is prescribed for the Muslim to perform Taraaweeh prayer
after ‘Isha’ on the first night of Ramadaan, which is the night on which
the new moon is sighted or the Muslims complete thirty days of Sha’baan. 

Similarly at the end of Ramadaan, Taraaweeh prayer should not
be offered if it is proven that the month has ended, either by sighting of
the new moon of Eid or if the month of thirty days has been completed. 

It is clear that Taraaweeh prayer is not connected to the
fast during the day in Ramadaan, rather it is connected to the onset of the
month at night in the beginning, and the last day of Ramadaan at the end. 

We should not say that Taraaweeh prayer is a naafil prayer
and it is permissible to offer it on any night and in congregation, because
taraaweeh prayer is limited to the month of Ramadaan, and those who offer
this prayer are seeking the reward that comes for praying it. The ruling on
offering this prayer in congregation is different from the ruling on
offering other prayers in congregation. In Ramadaan it is permissible to
pray taraaweeh in congregation on each night, whilst announcing it and
encouraging others to join, unlike qiyaam at other times, where praying
qiyaam in congregation is not part of the Sunnah unless it is done without
having the intention beforehand, or it is done for the purpose of
encouraging and teaching others.  So it is Sunnah to do it sometimes without
committing to doing it all the time or persisting in that. 

Shaykh Muhammad al-Saalih al-‘Uthaymeen (may Allaah have
mercy on him) said: 

Taraaweeh at times other than Ramadaan is an innovation
(bid’ah). For example, if the people wanted to gather to pray qiyaam in
congregation in the mosque at times other than Ramadaan, this would be an
innovation. 

There is nothing wrong with a person praying in congregation
in his house occasionally at times other than Ramadaan, because this is what
the Prophet (peace and blessings of Allaah be upon him) did. Once he
led Ibn ‘Abbaas, and once Ibn Mas’ood and once Hudhayfah ibn al-Yamaan, in
prayer in his house, but he did not adopt that as a regular Sunnah and he
did not do that in the mosque. 

Al-Sharh al-Mumti’, 4/60, 61 

Based on this, whoever offers Taraaweeh prayers before it is proven that Ramadaan has begun is like one who offers a prayer at the wrong time. No reward will be written for him, even if he is free of the sin of doing that deliberately. 

And Allaah knows best.