Yep agree again with everyone so far. One thing I would once again reiterate though....
Let them stay on your server if they choose to. If all has worked so far on your server without any hitches, there will be no reason for them to come back to you on any issues with the site and they probably won't be stupid enough to try to blame you for any issues on it. Just make it very plain that you will not be involved after that point in any changes on their site or server without charging a fee, and make it very clear to them (don't over do it so they think suspiciously though lol) that you will not be doing any editing whatsoever without their permission and request from that point forward.
This truly would benefit you if they are a paying hosting client as you would retain that income. This would also very much benefit you if they opt to return to you with the editing of the site, simpler, less hassle, and keeps the peace between you much better as well. Try not to take their choice to do things on their own so personally, I am betting it has nothing to do with you personally, more that they just want to do whatever it is they want to do without a middleman and possible fee.
I understand your frustration with how much work you've done without being compensated for it, but that was your choice in the end, and I would have done exactly what GregB said, the site would sit incomplete until the work is finished OR you give them a quote for the work that needs to be done if you're willing to do it, which you obviously were willing to do. You definitely need to give them the entire site, they didn't ask you to do what you did, you volunteered and now it has become a part of a complete work which would not be right for you to say they cannot have those items you did for free.
Don't do anymore work for free for them, that was your first mistake, but it was "your" mistake not theirs so don't punish the client for that.
Don't - give them all the original work as Paintbrush already said, and also make sure your ending agreement includes a provision stating they are not allowed to use those photos for any other reason as well as the part about you keeping the rights to the original work of those photos. That you Can do since they did not provide that part of the site it can be looked at as a single use item and give them a price per photo should they wish to use them elsewhere.
And I loved Paintbrushes idea on the full disclosure of all the work done with inclusions of statements of what has been done at no charge. Show them exactly what they got for their money, tie up the loose ends, and keep the door open should they want to return. Leave them on your server for your own ease of transition should they wish to return as well as the income it will provide each year. It may not be much, but it's money someone else will get if you chase them away right?
Ok I'm done ranting my book as usual hahaha. Good luck and I wish all goes well for you on this situation.