It all depends on your personal interpretation of 'early life'.
(this timescale is pretty rough BTW)
4 billion years ago, the earth was still soft and molten.
3 billion years ago, it has cooled enough for here to be 'oceans'.
2 billion years ago, erarly life was just "slime" that could spread.
1 billion years ago, some of it developed enough to be called 'plants'.
It took till a mere 300 million years ago before the first fish-with-spines appeared.
TO ME, that is "early" life.
I see your point though, we are a young planet in a young universe.
But life here "wasted" much of our 4 billion years.
What if 'life' on another planet didnt waste as much time being slime as we did?