In my hometown, swallows often nest on the beams of the main house. Every spring and summer, their family specially opens the door to facilitate swallows to fly in and out, saying that only those with good families will have swallows to nest.
I think many swallows fly down from your roof every day to rest there, which shows that this position is very good and the people and things around you are good. Don't drive them away.