The design IS a poor design and most vans will need to have the driver seat panel replaced in its lifetime. I have had to fix the panel in both the driver and passenger side of my T&C. The weak link is the forward fastener (the fastener to the far right in the above photo). It is plastic, and it slides over a metal piece that sticks out of the seat base. The connector between the trim panel and the fastener is extremely thin and will eventually break. Sure if you weigh more, it will accelerate the process, but cars made made in the 21st century should not be designed around people that lived in the 1940's where grown men were 5'6" and weighed 140 lbs.
O6DGC is correct and the problem is caused by the seat cushion which slowly compacts down over time as you accumulate more and more miles. The lip of the seat cushion begins to fold over the side of the panel when you get in and out of the seat, and the pressure caused by your weight on the cushion against the panel will eventually beak the forward fastener away from the panel. Gluing it will only hold for two or three cycles before it breaks again. You can buy aftermarket parts, but when I had this problem a number of years ago, I could only find the aftermarket part to vans without the lumbar support button, so I had to buy an expensive unit from the dealer. But because the seat cushion is the real cause, the new part will break as well, much quicker that the original panel did.
To fix the problem correctly, you will need to buy a new seat cushion. However, there is an alternative. You will need to get a fairly long stainless steel screw. Because it is stainless, it won't rust and it will look like a factory trim screw. Align the panel into the correct position and drill a hole through the front half of the panel, as well as through the seat base. You will need to make sure the holes align properly before you drill. Then, run the screw through the panel into the seat base. The panel will hold for good, it looks factory (especially if you do both driver and passenger sides) and you will not need to buy another panel or a seat cushion.