D&C 107

The Lord did covenant with Abraham -

"I will make of thee a great nation, and I will bless thee above measure, and make thy name great among all nations, and thou shalt be a blessing unto thy seed after thee, that in their hands they shall bear this ministry and Priesthood unto all nations;
10 And I will bless them through thy name; for as many as receive this Gospel shall be called after thy name, and shall be accounted thy seed, and shall rise up and bless thee, as their father;
11 And I will bless them that bless thee, and curse them that curse thee; and in thee (that is, in thy Priesthood) and in thy seed (that is, thy Priesthood), for I give unto thee a promise that this right shall continue in thee, and in thy seed after thee (that is to say, the literal seed, or the seed of the body) shall all the families of the earth be blessed, even with the blessings of the Gospel, which are the blessings of salvation, even of life eternal."  (Pearl of Great Price | Abraham 2:9 - 11)

The Lord explains to Abraham his seed "shall bear this ministry and Priesthood".  And that any one that receives the Gospel or is baptized and confirmed a member of the Savior's Church shall be considered Abraham's seed.  It is with his seed then that the right of the priesthood shall be held. 

This covenant was then extended to his grandson Jacob or Israel.  It was through this line and the House of Israel that this covenant would continue.  It is in this revelation that the Savior stated -

"The order of this priesthood was confirmed to be handed down from father to son, and rightly belongs to the literal descendants of the chosen seed, to whom the promises were made."  (Doctrine and Covenants | Section 107:40)

In my mind, the "chosen seed" is Abraham's seed or the seed or children of Israel.  The promises and rights of the priesthood then are reserved for Israel or the House of Israel which in our day, is the Church and baptized members.  The priesthood right then can only exists presently within the Savior's Church overseen by priesthood keys.

It is the House of Israel or family of Israel that will be linked together in the celestial kingdom.