Nephilim should be at least as ancient as the Steltek.
They use wormholes to travel and that does mean that when they do they are also travelling far to the FUTURE!! (Because Einstein's General Relativity Theory)
I'm not so sure that Nephilim are not Steltek, here is an hypothesis:
As the Steltek banned themselves away from the galaxy so many millennia ago, maybe some of them did not only retreat to the center of the galaxy (why should that be a safe place?) but happened to escape through the wormholes.
OK, Steltek at the center of the galaxy are noble and peaceful, just living their lives and trying not to interfere with anyone else; but some of their clans (or similar) have found themselves in a distant future still enraged by their old wars, and ready to attack anything in their sights.
-------
BTW: Steltek are NOT gone, How could they be still looking for their old tecnology if they are gone!?