Place the sweet potato in a medium saucepan and fill with enough water to cover the potatoes. Bring to a boil, reduce heat to medium, and simmer for about 20 minutes or until tender. Remove from heat, drain and set aside.
Melt butter over low heat in a small saucepan. Mix in the sugar, cinnamon, nutmeg and allspice. Add the apple slices, cover, and let simmer for 5 minutes, or until the apples are tender. Mix the apple mixture into the drained sweet potatoes along with the milk. Mix well using an electric mixer or just a fork until potatoes are mashed.
As a variation, you can mix in cloves. This is also delicious topped with brown sugar and pecans.