The whole point of the period from Rosh Hashana to Yom Kippur is to
engage in teshuva. Usually translated as "repentance", teshuva really
means "turn" (or "return"). Repentance is just one part of the process.
Teshuva is tied up with the concept of sin, or het. Het is a term from
archery, and it literally means "missing the mark".
When you see that you have missed the mark, you must turn yourself so
that you aim can be true again. Teshuva is a five-step process: you
must confess the sin; if the sin was committed against another person
you must ask that person's forgiveness; you must cease to commit the
sin; you must regret the sin (and make restitution if possible); you
must resolve never to commit the sin again.
The mark of true teshuva is that you actually do change your behavior.
This is not ultimately about how you feel about your actions -- it is
about the actions themselves.
Rabbi Eliezer said: "Repent one day before your death." His disciples
asked him, "Does then one know on what day he will die?" "All the more
reason he should repent today, lest he die tomorrow" (Shabbat 153a).