I don't see the rail in the lawn using the Google Maps Street View at 48 Roque Bluffs Road.
However, check out: 175 Washington St, Eastport, ME - no rail visible, but there is a boxcar.
My topo maps show a number sidings and one large yard between Ayers junction and Eastport including two spurs that crossed route 190 in Quoddy. Street View doesn't show anything, but the only way to know for sure is to check.
Let me know if you want the topo maps showing the sidings, spurs, etc.
-Ed