Preheat oven to 425°. Place sweet potatoes on a rimmed baking sheet and roast, turning once, until tender, 35–45 minutes. Let cool slightly.
Meanwhile, whisk garlic, oil, parsley, sweet marjoram, and ½ tsp. vinegar in a small bowl to combine; season herb oil with salt and pepper.
Halve sweet potatoes lengthwise. Using a large metal spoon, scoop out large pieces of flesh and place in a large bowl; discard skin. Add remaining 1 tsp. vinegar and gently toss to coat; season with salt and pepper.
Transfer squash to a platter and drizzle with herb oil. Crumble goat cheese over.