Proposal for adding a build step

[kaicataldo]

Originally from #536 (comment).

In pursuing #536, I would also like to revisit the idea of adding a build step to our site. We know it would be a prerequisite for any of the options we've discussed in #536, and separating that task out into its own chunk of work feels like a much easier/cleaner way to do things. Besides unblocking us from using a different static site framework, I believe we would get the following immediate benefits:

1. When I first proposed this, we weren't pulling data from Open Collective's API. The build step could now automate running less, bundling the parser and demo, and running the script we use to pull data from Open Collective. Currently these are run by individuals before committing, which is extra overhead and as a process is error prone (someone might forget to run the script). Automating it would ensure it happens on every commit. Edit: @ilyavolodin corrected me below to say that the Open Collective scripts are actually run on the hour and should be included in this.
2. We would get to dogfood ESLint in a more modern web application.
3. I believe this would make it more approachable and enjoyable for our community to contribute to the site (particularly the demo).

Here is how I propose we move to having a build step:

1. We could use the current repository, but GitHub pages has the unfortunate limitation of only allowing hosting from the master branch for a user or org site (one that ends in *.github.io). We could create a separate src branch, but that feels confusing and would require anyone working on the site to know that master is essentially a build artifact. If we were to create a new repository called eslint/website, we could treat the master branch as the source and publish to a second branch called gh-pages. This way, master is always the source of truth.
2. If we do create a new repository, we would have to update the release script to also copy documentation files into that repo in the interim.
3. We set up Travis (as described here) to run whenever a commit lands in master, running the build and publishing it to the gh-pages branch.
4. Update the build script to bundle the demo and the parser with Webpack, run less, and run the script that pulls from the Open Collective API.
5. At this point, we can evaluate how this is working and perhaps take it to a TSC vote to determine if we want to move forward with a build step.
6. Once we cut over completely to that repository for the website, we would have to update the release script once more to only generate docs in the new repository.
7. Deprecate the current repository in favor of the new one.
8. Edit: We would also have to transfer issues over. It doesn't look like there's a built-in way to do bulk transfers, so we'd probably want to use an community existing tool (looks like there are a few) to do this.

If nobody is against this, I'd love to go ahead and make the new repo and get it working. Once that's done, we could evaluate if we want to continue with that or not. Deciding to try it out does not preclude us from continuing as we have been in this repo.

[ilyavolodin]

I'm not opposed to having a build step, but with the current configuration of the site I'm not sure what it's going to gain us (other then added complexity). In my mind, I would hold off on adding build step until after we have a new design ready.
If you do work on this, please keep in mind that Open Collective API calls should absolutely not be part of this build step, as we run those every hour. It's very important to update the site as soon as possible when we get a new major donor. But team updates can probably be moved from separate jenkins build into site build.

[kaicataldo]

@ilyavolodin I misunderstood how we were getting data from Open Collective. Do we have a cron running these somewhere?

[ilyavolodin]

Basically. It's a jenkins job that runs hourly.

[kaicataldo]

Also, to give some context around my motivation, I also wanted to mention that the site redesign discussions have stalled (since about January), but I think we have enough of it designed out to get working on it. I have some time now and was hoping to make progress now to help unblock future work.

[ilyavolodin]

As I mentioned, I'm not against adding a build step, I just don't really know what we would use it for in the current setup of the site. If you foresee major changes with the next version - sure, it sounds fine to me.

[nzakas]

Sorry, still struggling healthwise, but wanted to add my 👍. I think right now the way CSS and the demo are managed are difficult for new contributors to figure out (I've had trouble, too), so I think a build step would be helpful.

I'd also add that this might be a good time to consider hosting the site on Netlify. I've never had good luck trying to build from master and deploy to gh-pages, and Netlify would automate the build step for us for free. Clearly not a big blocker, but wanted to bring it up as it seems like this would be a good time to consider it.

[kaicataldo]

I'll take a look at Netlify and report back - if we can use the existing repo, that might be an easier process than the one I've proposed above.

[ilyavolodin]

Netlify would allow us a lot of flexibility, but I think we need to be careful with it, since it does have a limit on the free account, and I think we will potentially hit that limit very fast.

[kaicataldo]

Netlify offers a free plan to open source projects which looks like it has the Pro tier features with unlimited members, so we should be okay if we decide to go that route.

I came across this issue, and it looks like we are already hosting some form of the site at https://eslint.netlify.com. Do we already have an existing account that I should use to try this out?

[kaicataldo]

Reporting back on Netlify. It was really simple to set up and I think this is the way to go. Here's the site being built from master with a free plan I just created.

Here is their Open Source Plan Policy page.

Pros:

• We can build directly from master and won't have to create a new repo, update the documentation generation scripts, or have to worry about transferring issues over. We could rename the repo, but that doesn't seem necessary unless we think the current name signaling a GitHub Pages hosted site is confusing.
• Netlify will provide us with a free open source plan that allows unlimited members
• We can set up SSL through Netlify (this isn't an issue we currently have, but it's nice to know it won't be an issue if we choose to use this service)
• Per PR branch deploy previews (this sounds really useful for reviewers)

Cons:

• Another service (albeit free) that we'll need to sign up for

One thing to note: the free open source plan requires that we link to Netlify on our home page, which I think is more than fair for the free services they would be providing to us. They actually provide premade badges, so this is as simple as copying and pasting some markup.

Here's what I'd like to propose:

1. We publish the site to Netlify in its current state (only running Jekyll during the build step)
2. We wait for Netlify to grant us an open source plan and I can add whoever is interested to it. I think we need at least one or two other people who have access to the team as a backup.
3. We point eslint.org (I'm not sure who on the team owns it, but this step will require their help!) to the Netlify hosted site
4. We unpublish the GitHub pages site
5. We now have a build step and we have the freedom to update things as we see fit!

Please let me know how this sounds - thanks!

[ilyavolodin]

Additional pros for Netlify is that the site will no longer be fully static. We can use their dynamic functions to provide information to the site without having to re-run build (we can probably use this for team members and maybe even for sponsors). Also, Netlify is flexible in terms of framework usage. So we can use React or Vue later to re-write the website. Netlify CMS also can help in some cases.

[j-f1]

The https://eslint.netlify.com site appears to be owned by @alberto.

When I got a Netlify open source plan for @typescript-eslint, it only took them like 4 hours to grant me the plan.

[kaicataldo]

Quick update: I reached out to @alberto and they have taken https://eslint.netlify.com down. I’ve also been in touch with Netlify support and we have been granted an open source plan if we would like to use it.

[kaicataldo]

One more update: the TSC voted today to move hosting to Netlify. I am in the process of inviting TSC members who are interested in helping out, and I'll be reaching out to the JSF to inquire about pointing the domain to the new site. Additionally, I've changed the domain to https://eslint.netlify.com/, now that it's free.

[ilyavolodin]

Just to clarify, our domain is not pointed to github. It's pointed at cloudflare which caches our site. So we might not need to move domain, just reconfigure cloudflare. OpenJS Foundation owns that configuration and account as well. It was originally setup by @kborchers (I believe).

[not-an-aardvark]

Oh, I'd been assuming that we would no longer be using cloudflare as a result of the switch to netlify. I'm fine with either option, though.

[ilyavolodin]

I'm also fine with dropping cloudflare. Actually Netlify is saying that you really shouldn't use another CDN in front of it. I don't know if it will result in any problems, but it's possible.

[not-an-aardvark]

If we do drop Cloudflare, we need to make sure we set up HTTPS with Netlify, or the site won't load. (I'm pretty sure this is possible -- I'm just mentioning it in case it requires additional configuration.)

[j-f1]

@not-an-aardvark It’s not too hard — Netlify will set up and automatically renew a Let’s Encrypt certificate with the click of a button. If you’re feeling really fancy, I believe there’s an option to upload a custom certificate too.

[kaicataldo]

I think it makes sense to drop Cloudflare and use Netlify’s offerings. According to Netlify’s docs, it looks like we should just be able to create an A record pointing eslint.org to Netlify’s load balancer’s IP, and then once it verifies the domain configuration Netlify will be able to generate a cert for us.

[not-an-aardvark]

Works for me -- it might be worth announcing site maintenance on twitter in case the switch causes some downtime while the cert is being generated.

[j-f1]

There should be downtime since the site has an HSTS header that specifies that eslint.org should be hardcoded into browsers so it’s always visited over HTTPS.

[kaicataldo]

I can reach out to our JSF contact and see if we can find a mutually convenient time for them to make these changes. That way, I can tweet this out as well.

[kaicataldo]

There are a few kinks that need to be resolved (that look like they might work themselves out as the DNS changes propagate), but we have officially cut over to Netlify!

We should hold off on changing this repo until we're sure DNS changes have propagated, but then I'll be working on adding bundling into the build step as discussed.

[kaicataldo]

One more thing we should discuss here: I've discovered that we can't actually unpublish this GitHub pages site unless we delete the master branch or delete (I think that means we can rename?) the repo. That means that once we add a build step, the site hosted at https://eslint.github.io will actually be broken. Thoughts on renaming the repo to eslint/website?

According to GitHub's documentation, it looks like it'll forward any traffic even if we rename it.

In addition to redirecting web traffic, all git clone, git fetch, or git pushoperations targeting the previous location will continue to function as if made on the new location.

One thing I'm not sure of is if we can rename it back to eslint/eslint.github.io, but since we plan on adding a build step, we probably want a non *.github.io repo anyway, so I don't think it's a concern for us.

[j-f1]

I’m pretty sure that as long as you keep the CNAME file, GitHub will automatically redirect eslint.github.io to eslint.org.

[kaicataldo]

Oh, interesting! I do think it would still be better to just rename the repo, but that's a good thing to keep to in mind.

[kaicataldo]

TSC Summary

We now have a build step! The final thing to decide here is if we want to stop publishing to GitHub Pages.

We can't actually unpublish the GitHub Pages site unless we delete the master branch or delete (I think that means we can rename?) the repo. According to GitHub's documentation, it looks like it'll forward any traffic even if we rename it.

In addition to redirecting web traffic, all git clone, git fetch, or git push operations targeting the previous location will continue to function as if made on the new location.

One thing I'm not sure of is if we can rename it back to eslint/eslint.github.io, but since we plan on adding a build step, we probably want a non *.github.io repo anyway, so I don't think it's a concern for us.

TSC Question

Should we rename this repo to eslint/website?

[kaicataldo]

The TSC voted today to rename this repository to eslint/website.

[kaicataldo]

I have changed the repo name, made sure publishing to GitHub Pages is turned off, updated the Jenkins job, and made a PR to eslint/eslint to update the release script. Am I missing anything?

[kaicataldo]

The eslint/eslint release script has now been updated and all the Jenkins scripts have been updated to use the new repository name while clone (though they should have worked regardless due to GitHub redirecting the requests).

I think we're done here! Thanks everyone for all the input!