Video Transcript: Unlocking Web Development Team Capabilities with AI

All right.

Back in business.

Let's see. I'm in business. All right. Yeah, thanks for having me. Appreciate having the chance to be here and speak with you all and present on this. And thank you, Ron, for [CHUCKLES] saying my unwieldy title for this. I do have these slides available at this Bitly link-- bit.ly/UMassUnlockAI. I make reference to tools and potential options for AI and further reading that you can follow, so you can get these slides there and check them out at your own convenience.

All right, so you already heard who I am-- Director of Web Development at MIT Sloan. So mitsloan.mit.edu is the main website my team and I work on, among some others. If you're curious, for the technical minded out there, it's a Drupal 10 website. If you want to know more about me, you can find me at mike-miles.com or all over the internet at @mikemiles86, so.

All right, what I'm going to talk about today are a few things. First, just a general overview of generative AI. I'm sure everybody out there who's watching has had experience with generative AI, has played with it, whether it's ChatGPT or Copilot or what have you, but I just want to level set just for the concept of this talk.

And then we're going to talk about some of the benefits of using AI for programming and web development in web development teams. And then talk through ways you can use AI to enhance your web development teams and the roles that they fill and the value that they provide to the university. Then, some considerations and cautions you want to take into account when you're looking into using AI tools or AI-generated content.

And then I have slides for additional resources. I'm not going to cover them here because they're just links, but those will be available for you.

So generative AI, again, I'm sure everybody here has played with it because it's been the talk of the town for the last two years. And really, it's a subset of AI. AI itself is a huge area of research covering many facets. But generative AI is really just focused on the ability to generate human-like content. Getting human-like input and producing, based on algorithms, what it thinks is the best human-like output for that input.

Now, these generative AI models that you've probably played with, like ChatGPT, or Microsoft Copilot, or Claude, or what have you, they're trained on large amounts of data-- terabytes of data. I mean, large to humans. Not very large to AI, really. Famously, everybody is probably aware that OpenAI has scraped the whole internet to train its models. And so it knows, like, everything of human knowledge that was mostly publicly available. They say they didn't target privately available data, but who knows, really?

And the way it works, at a very high level, is it uses complex pattern recognition algorithms to predict output. They're word calculators-- the way I like to think of them. That's a very crude way to put it. But you give it a set of words, and these generative AI tools, whatever they are, analyzes those words and figures out what the next best sequence of words are to complete that context or statement. And that's what they do.

How they do it is very murky still. Researchers are starting to gain insights on how these AI models actually work, which-- you can go look at the research, as I find it very fascinating. But for our consideration, they're word calculators. They're figuring out the best thing to provide to you based on what you ask it.

So when we talk about generative AI for web development and programming, the two go really well together. And you've probably seen that in the emergence of the marketplace for AI tools. And the reason for this is because code, whatever programming language, whatever web frameworks you're using, they're full of patterns, and AI models like patterns. That's what they're driven to do-- is to figure out patterns.

So since any model has consumed all of the public knowledge available out there, every programming language, especially for the web, like JavaScript, PHP, Python, .NET, those have structures and lexers and information about how they can be used and the logic for how to write functions and functionality and best practices.

That's been consumed by these models, so they know all the best practices for all your programming languages, for all your frameworks. They know what's expected. They know the rule sets for them, probably better than your most senior developer.

So when using generative AI for programming and web development, I like to think of it as the ability to free up time for higher-level problem-solving. So your role at UMass-- if you work on the website, if you're a programmer, if you work in IT, if you work in communications marketing-- your role is not to be the expert on how to, say, use your CMS. I mean, that's great for your role and that's the tool you use.

But your role and everybody's role, in my understanding, for any university is about being able to use your skill sets to promote and distribute the mission and goals of the school for your department, to get that messaging out there to your audience, be that students or faculty or researchers or whoever you're trying to reach. To teach them the mission and the values of the school, and to really take the goals of the school and use your skill sets to do that.

So for web developers, if you're working on the school's website, you want to be able to think more about what you're trying to solve for the school with the web tools and then have AI help you use those tools more effectively to do it so you can spend less time doing the nitty-gritty tasks of writing out functions and variables and figuring out the best algorithms to use for your functionality. So you can think more of the high-level problems you're trying to solve and then use AI to help do that.

Now, you still need your technical skills. I'm not one to vote for getting non-technical people writing all the code for web programming. But to augment it, to use another tool to expand your capabilities and increase your capabilities, is definitely a value here.

Now, there's a huge market of developer-specific AI tools you've probably seen emerge. It's one of the fastest-growing markets for AI. I mean, AI is touching everything, but for web development and programming in particular, because of the use of patterns and how well it can be applied to it.

So I'm going to talk throughout this presentation. I'm going to name a few tools that I've looked into. One of the things I like about this talk is every time I give it, I get to update it because the market is changing so fast. So I have some relevant updates of potential tools that can help fill gaps in web development teams.

So I like to think about AI as a teammate. I mean, AI is a tool, but if you think of it as a teammate, that can help you decide how you want to use it to augment your web teams. So as a teammate, AI-- it's going to be your most knowledgeable yet least informed member of your team.

Most knowledgeable-- as I said, it's consumed all the data. All these models know all this information about pretty much every programming language, every framework. It knows the technical nitty-gritty of those things and how to follow those patterns, but it knows nothing about how you want to use those to reach your goals, the university's goals. So it's most knowledgeable in terms of technical knowledge, least informed about what you want to do with those. And that's where you have to help fill in the gaps.

And this was about six months ago, or a little longer. Someone put it to me like this-- think of these AI tools as artificial interns. If you were given an unlimited number of computer science interns, what would you have them do? And I think this is a great way to frame how you could think about using AI tools in your development processes, your web development processes, because how would you give tasks to an intern? What would you do with the output from an intern?

You wouldn't tell an intern a very vague sentence of, like, build me the website, and then take whatever they give you and just release it to the world with the UMass logo on it. You would review everything. You would be very specific in the details you give them, and you'd check what they produce before you release it to the world to represent the university.

So thinking about it in this way, for any AI tool you use is that it's an intern, so how specific do you have to get with what you're going to tell it? And then what are you going to do to review that work and further work along with that artificial intern? So I think this is a very good framework for putting in all these generative AI tools and how you might use them, especially in web development.

Again, adopting AI tools as a pseudo teammate to expand your dev team allows you to focus on solving the project needs, not necessarily the nitty-gritty technical needs. You don't have to think about, well, what's the best way for our developers to implement these features or functionalities or campaigns? It's, what do these campaigns need to deliver? What do they have to produce? And then how can we use AI to help us build that with our technical knowledge in-house?

And in my experience-- I mean, I've been working in higher ed for five years-- but higher ed web teams usually are small. My team at MIT Sloan, we're only five people, and so there's a lot you have to do when you work-- we work out of the Office of Communications. When you work in marketing and communication side, there's a lot more you have to focus on than just writing code and just building your CMS and keeping it up to date.

There's a lot of other dev-adjacent roles you have to fulfill, be that user training, content writing, UX design. All these things are important tasks that take away from the building product and producing output. So having AI fill in those gaps if you have a small team and you don't have all these roles available can be really helpful and really help speed up what you're able to do and increase what your team is able to do.

Now, just like any member of your team, being able to talk to your AI teammate is the most important thing. Communication is always number one, whether it's with our stakeholders, with our students, our faculty, staff, the president, or our AI agents. We want to be able to tell them exactly what we expect of them or what they expect of us, I guess, but be clear in our communication so everybody understands what's expected.

Now, still today, most AI interactions happen through prompts. You type in or you even speak your prompt, what you're asking the AI to do, and then it calculates what it wants to output. So working on your prompt engineering skills, I think, is an important skill to develop for anybody working on the web or in programming.

So if you're not already working on playing around with how to best produce prompts, keep working on it because I think in the next few years-- already we're seeing this-- but in the next few years, it's going to be expected that developers know how to work alongside AI to produce output.

I like to attribute this back to early days of development, like decades ago. You would write low-level code, and now we write in higher-level languages that abstracts that. And we use DevOps processes that abstract deployment processes, things that we don't need to do by manually by hand. And AI is just another tool, and the way you work with it is through prompting.

Now, I've updated this slide because recently, in the last two months, I've changed my personal way I format my prompts for working with generative AI tools. And I learned this new method-- well, new to me. It's C.R.A.F.T, which is a acronym, from this YouTube channel, Lawton Solutions, which they have YouTube videos about working with AI.

And actually, this video linked to-- when you view these slides-- it's about how to use AI using the C.R.A.F.T format to generate a prompt to give to AI so you produce better prompts. But what C.R.A.F.T stands for is context, role, action, format, and target audience.

And going back to thinking about AI as an artificial intern, thinking through all this information you have to provide in a prompt is really helpful, You want to give them the context of what they're trying to accomplish, the role-- in this case, the AI-- what role it's playing-- the actions you expect them to take, the format you expect the output, and then who the output is for. Providing these levels of detail for all your prompts will allow you to produce more specific prompts for your problems and your needs so you don't get generic answers. You get very specific answers.

As an example of this-- so this is going to be a mouthful of a prompt, but I hope it's very relevant. So let's say we are going to write a JavaScript code to retrieve JSON data by making authenticated post to an API endpoint. You, being the AI, are an expert JavaScript programmer who writes well-documented and readable code.

So I wanted to note here just the use of a lot of explicit detail words-- expert, documented and readable code. This helps guide the AI on what type of output to produce. Especially with code, this is important-- saying, like, you're an expert developer-- because since it's consumed a lot of data, including all of Stack Overflow, there's a lot of poor code out there.

There's a small amount of really, really great code out there, there's a bigger point of bad code, and there's a big area of mediocre code. And so putting in "expert" tells it to focus on the best code instead of the mediocre-- the median code. Continuing with our prompt.

Number one, ask additional follow-up questions to make sure you are clear on what the code needs to accomplish before providing solutions. Number two, when you fully understand the requirements, write out three JavaScript solutions. So I like doing this for actions, especially the first thing of saying, ask me questions before you do anything. Again, this is something you would tell an intern. Here's what I want you to solve. Ask me anything that you need to know more of before you get started so that you produce the thing I'm expecting.

Then, further going on with our prompt, write out the JavaScript code for each solution, and for each, provide a list of pros and cons of the approach. Then finally, the output is for a junior JavaScript developer. So this is a pretty detailed prompt. And if we break it down to that C.R.A.F.T format, we see the context, we see the role we want the AI to take, we see the actions that we want it to take, the format of the output we want, and who the output should be for.

So even if you're not junior, I like to put in I'm a junior developer for the output for the target audience because then it provides more description and details about what it produced. So this will produce something very specific to solving this problem.

And before that, the AI will ask me, like, hey, how are you authenticating? Is it a basic auth? Is it token-based? What do you need to do with the JSON data? Do you need to store it? It's going to ask these follow-up questions before producing any code. So this can produce something that's very useful.

All right, so moving into roles that AI can play, the first one I like to go with is AI can be a rubber duck for web development teams. If you haven't heard of rubber duck developing, what this concept is is if you are a developer and you're working on code or you're trying to solve a bug and you're just stuck-- you're thinking through it-- you can't figure what the problem is-- the idea is saying the problem out loud, talking through it out loud, be it with another person, an AI tool, or a rubber duck on your desk.

Which, I've had a rubber duck on my desk for years. Actually, right now I have a bobblehead of myself, so I talk to myself a lot. Saying it out loud changes the way you're thinking about it and helps you actually find the solutions to the problem. So I think AI is a really useful tool for developers of being a rubber duck. It can help you spark new ideas and approaches to problems that you're dealing with.

If you're just trying to solve something the same way and you're not getting the results you want, talking to AI and say, hey, this is the problem I'm trying to solve for. These are what I've already tried. Since it knows all the best practices and all the methods for all the frameworks out there, it may produce different options and solutions. Now, they may not solve the problem that you're actually dealing with, but it's going to get you to think slightly differently and approach a problem in a different way.

This allows rapid iterations and brainstorming when you're planning, say, big projects. If you get a request to add a whole new marketing campaign to the UMass website that has all new functionality, I mean, that's a big chunk of work to take on, and you want to figure out how to plan it out. You can talk to AI and say, hey, this is what I'm trying to accomplish. Help me break it down to smaller steps in tangible tasks.

And that brainstorming can help you iterate a really useful plan. It's not going to get everything, but it's going to work back and forth with you to do that.

And then I think conversational prompts with AI, like a back-and-forth conversation, asking questions and having it ask you questions, this helps practice explaining technical ideas and concepts clearly. So if you're a developer like myself, we're not as well known for our ability to clearly communicate to non-technical audiences. Now, I'm not saying that's true for every developer, but it's a general use case.

And so if, say, the website goes down or there's a page that has a big glaring error on it and the president comes to you, or the provost comes to you, and says, What's going on here? and you give a big, detailed explanation, they're not going to understand it. It's not their job to understand that piece. But because they don't understand it, they're not going to feel confident that you can resolve it easily.

So practicing explaining technical ideas to AI helps you get better as a developer of explaining technical things to non-technical audiences. Going back to how you generate prompts, you have to clearly explain what you're looking to achieve with your prompts, meaning you have to be able to explain what you want your functionality to accomplish. And you can't use too much technical details because it may get lost, so you want to speak, quote unquote, "clear English" to it.

I think this is also helpful-- from a rubber duck perspective in all these three bullet points here-- to help deal with imposter syndrome, if you've heard of that. That's the feeling in development-- I mean, it can be in other roles as well, but especially in development-- when you feel like you're not supposed to be where you are, that people have too much faith in your capabilities as a developer, and when you self-doubt what you can do.

And one of the problems with imposter syndrome is that that limits what you're willing to ask to your teammates, to your colleagues, just from a fear of them finding out you don't know what you're talking about. But AI doesn't care about you, and that's a pretty blunt way to put it, but it doesn't. So you can ask it whatever questions you want, and you don't have to be worried about what it's going to think about your knowledge or your skill level as a developer.

You can ask the most mundane things that-- I've been working in web development for 18 years, and I could still ask, what's the proper order to put things in for a POST request? Or something. Or-- I've been working PHP and Drupal mostly-- what's the right way to write a Drupal module? Things that people would probably expect me to know at my level, which I mostly do. But if I want to reaffirm my knowledge, I can ask AI and not worry about what people think.

So I think it's helpful as a rubber duck to help with imposter syndrome and get comfortable asking what questions you need to ask to be clear about what you want to do.

So some tools for rubber duck developing with AI-- phind.com is a interesting one to try out. It offers code suggestions in your code, a lot like GitHub Copilot, but this is all in-browser. It can even do a pair programming capabilities. I think this is now, though, at a paid tier. But you put in your code, and it'll step through the code with you, asking you questions that you can explain. So that helps with that conversation piece.

Uizard, slightly differently, can be used for a rapid web and mobile UI prototyping. So following up from the last talk, having a little UX is better than no UX. If you don't have the finances or the capabilities on your team to, say, hire an agency to help with UX, to develop UI, and you just have a small team of developers-- you have an idea of what you want to accomplish-- you could draw out a paper napkin sketch in Uizard to upload that.

And it can help produce accessible code and help produce a UI that you can refine and work through that looks more professional than you might be able to do on your own. Again, that's not going to solve all the problems, but it can help speed things up and help augment what your team is already able to do.

All right, the next role is AI can be a reviewer-- reviewing code, reviewing documentation, reviewing details and needs and requests. These are all important pieces about web development. So as a reviewer, you can use AI to help save time by flagging common problems early.

If you are lucky enough to work with other developers on your team, you may have a system in place, a workflow in place where you produce a pull request or a merge request for your code, and someone else reviews the changes. That's helpful because if you have a more senior developer looking over code changes, they can flag problems. Or junior developers, if you have them looking at your code, they can ask questions.

But there may be times where the team is really busy and you don't want to necessarily bother people to look at your code and find things that you should have caught already. Not to say that it would make you a bad developer in any way. We all miss things. So having AI in place and AI tools that can review your code, be that in your IDE or in your deployment workflows, and flag those known best practices, flag known problems, and catch those. Kind of like a linter-plus.

That can be really helpful to speed up that time before you have to ask somebody else for help so that they can continue doing their work as well. Again, since it knows all this knowledge, you can use these AI tools for your code to suggest improvements based on those best practices and standards and known design patterns.

Depending on the tool you use, if you have certain programming standards or even design patterns for the website for the school that you want to provide to an AI tool to say, use these when evaluating this output, say, for visual regression or something like that, these tools can help flag those things and catch them again before they go out or go to somebody else.

I like to use AI to review complex error logs to offer insights into root causes. Any developer out there, anyone working on the web, you've experienced an error somewhere. I mean, it's worse when end users see it, but when you see it, it's slightly better. But you maybe get a whole page of a stack trace of what's caused the problem. And you could parse through that and figure out the solution. I'm sure of it. But that takes time, and if you're dealing with a critical issue, you may want to speed that up.

So you can take that stack trace, give it to an AI tool, and say, hey, summarize what this is telling me is going on. And again, it may be able to more quickly point you into the right direction of where to solve the problem. It's not necessarily going to know all the context of how to solve it, but it could say, hey, this is telling you that there's an error in blahblahblah.php at this function. So it can just help parse through those errors a lot quicker.

Two tools I like for this-- Codiga, which provides static analysis and code reviews. You can hook it up to many of the popular deployment pipelines out there. So it can take a pull request or a merge request or code changes. It can run through it for static analysis and provide AI insights to code reviews to say, hey, here's how you might want to refactor this section. And actually, just recently, GitHub, with their GitHub Copilot, just implemented some new code review capabilities of Copilot they can use GitHub actions for.

PrivateGPT is an interesting one where that can run completely locally on your machine. It can run on a laptop, and that can consume and prompt only on local files for your project. You'll have all the knowledge of a model, but it will use just the context of your file base of your project base to answer questions.

I like to use this. I've actually downloaded our complete user guide for our website, and I've tried it with PrivateGPT, where I can say, hey, what does the user guide tell me about how to write an article? It will provide me information so I can find out gaps of that knowledge there so I can review what our user guide might provide and might not, and we might want to update that.

Moving along with that, AI can be a documenter. Documentation-- another dev-adjacent task that all teams should be doing. Not every team does. Developers-- I speak for myself again-- are not the best at writing documentation. It's not our favorite thing to do, but it's important for everybody involved-- for yourself, for other developers, for your end users, for your product managers, for your product owners-- so that everybody knows how the system works, what they can and can't do.

So AI as a documenter-- again, if you use a tool that can plug into, say, your IDE, you can ensure that your code is well documented. So you can use AI to generate code comments that are more helpful than say, this is a variable that does x, y, or z. We all get lazy with our comments sometimes, and that can be detrimental to the team-- either other members on your team who are reviewing your code or yourself in six months, when that code is the source of a bug and you can't really understand what it's doing because of your poor commenting.

So using AI to help you generate robust comments in your code can be really helpful so that later on when you have to review that code, you know exactly what it's doing, exactly what that function is doing, and how it's translating data.

I like to use AI to document and explain third-party code, legacy systems, or documents. I would be really interested if anybody out there has ever inherited a legacy system that was well documented. In my experience, legacy systems are not ever well documented. So you may come across some functionality that is just a 300-line function that you don't really understand what it's doing. You can use AI to help explain that to you.

Or if you're dealing with any third-party code that has been minified or obfuscated, that's hard for humans to read. The AI can read it just fine. So you can provide that code to the AI and say, what is this doing? Help me understand what this section of code is doing. So you can better understand it and improve it or work on any problems that it's causing.

And then you can use it to convert technical specs, ticket details, et cetera into understandable documentation for your users. So if you work, say, with a ticketing system for your development-- my team, we use Jira-- you may want to take your technical specs that you use for building out your functionality and turn that into some sort of user documentation.

Now, you can do that. And again, developers can do this. But you may want to use AI to help get you further steps faster to say, turn this into documentation that an end user will understand how to use this system. And it may not get it all right, but it's going to get you in a starting place. And it's going to get you a little further along a little faster and save that time.

Some tools I like for this-- GitHub Copilot. My team uses this as a VS Code plugin, actually. And so we use this to analyze code. It can write inline code documentation. It can write code based on code comments. So you're kind of documenting the functionality you want, and it can generate code which then you can review, and edit, and update.

Cursor, which is an AI-first IDE, which is based on VS Code. I find this very interesting approach. It's like halfway to vibe coding, if you've heard of vibe coding, but it's all based on prompting. That's what expects you to interact with the IDE first. And so this really gets you to the habit of documenting and writing out what you want the code to do, having AI generate it, and then you work with the AI to refine that code.

More general-- AI can be a time saver. That's what we're all looking to do. That's the point of this-- is that as AI is a tool, even though I'm talking about it as a teammate, we want to use it to help save time so that we can accomplish more so that we can get better results with less resources and less money, ideally.

So as a time saver, you can use AI to generate realistic data, realistic content, realistic images for use in your lower-level environments, be it your dev or QA environments or your local environment. Depending on the work you do, you may be dealing with sensitive information. Say you're dealing with student enrollment information or faculty data-- you don't necessarily want to copy that down into a local developer's environment on their laptop for security purposes.

But you could use AI through prompting to say, generate for me 300 student records with x, y, and z fields. And that information is going to be completely made up, but if it's the right structure, you can import that into your database and work with pseudo people. On my team, our website-- the MIT Sloan website-- most of the content is editorial content based off our faculty research, and our editorial team writes it up as articles.

So when my team is working on new features and functionality that we want to test for articles, we can use AI to generate an article, let's say, on climate change. I don't necessarily care the content of that article. I just care that it's actually realistic-- more realistic than lorem ipsum. This is the place where hallucinations from AI is OK because you don't care about what exactly it says-- just the structure mirrors something that's realistic, more so than lorem ipsum.

Or another great example-- for any web developers out there, I'm sure you use placeholder images, probably of kittens. And if you show that to stakeholders, they get a little weirded out by it sometimes. They don't always understand that it's just a placeholder. So you could use AI to generate images that are representative of the realistic content that's going to go on those pages or go in that campaign. You can still have FPO for placeholder image, but it helps them conceptualize the content better than saying, just ignore the kittens. They're just there for a placement.

You can use AI to scaffold boilerplate code for custom development if you have to do any of that again for any of your frameworks that you're using. I know, for example, that the UMass Amherst website, at least the homepage, is built on Drupal. And so if that team needs to write a custom Drupal module and maybe they're not as experienced doing that, you can use AI to generate the boilerplate for a module and say, hey, I need to create a new Drupal module. Can you help me scaffold out the files? And it can provide that custom output that matches the best practices, and then you fill in the details of the features and functionality.

Some tools for this-- ChatGPT is really good at generating data. It has a lot of methods. It can generate a lot of different types of information. Microsoft Copilot as well. Any of these major generative AI tools that you prompt through chat can generate realistic data. You can even ask it to format it for you as a database query that you can insert into your local database.

If you do any WordPress development, CodeWP is an interesting one that can scaffold WordPress plugins. Now, in my day-- my team does manage a WordPress site as well-- and back in my earlier days of development when I would write WordPress plugins, my method for doing it was looking at the way a plugin I was using already was structured and to copy that structure.

And then the next time I had to write a plugin, I would copy that structure. And it was like degradation of copying. So using a tool like CodeWP where it sets up the plugin structure, following best practices every time, can be really helpful.

All right, so some considerations when using AI. I think this is very pertinent for higher ed, is to think about what rules and regulations and what cautions we need to think about with our AI tools. First and foremost, models only produce output that looks correct. They're word calculators, basically. Very sophisticated ones, but still, they're just going to produce what they think looks correct based on the rule sets they know. This doesn't mean it is correct.

So just like if you gave an intern a task to produce some code for you, you're going to review it. You're going to check it to see-- is it secure? Does it produce the right output? Does it solve the problem I need it to do? You want to do that with anything you have AI produce, especially with content. If you're using AI to generate content, especially in higher ed, you want to really review that because that's representative of the university if you put that online.

So just keep that in mind that it only looks correct. Luckily, with code, "correct" means it either works or it doesn't work, usually. So that's easier to test. You can even use AI tools themselves to review what you have an AI generate. You can have AI generate a function, give that to another AI, and say, hey, is this secure? Are there any problems with this code? But keep that in mind-- it only looks correct. It doesn't necessarily mean it is correct, so you need to check its work.

AI-generated content is not copyrightable in the US, and there's a bunch of lawsuits going on right now about this. And I am not anywhere near being a lawyer, but that being said, if you work with any proprietary code bases and you use AI to generate code that you then put into the proprietary code base, or you're using some proprietary license, like the MIT Open License, what does that do to the licensing of your code? What does that do to the ownership of your code and your systems?

I have no idea, and you probably don't either. So you want to be mindful of that and definitely check with the legal teams and be like, what's OK for us to do and not do here?

Any data you provide to third-party AI tools might be used for further training. For example, OpenAI with ChatGPT. If you use the free version, anything you put into it, they say in their terms of service they can use it for further training. So you want to be mindful of what you're giving to these third-party systems.

If you're very concerned about providing a huge section of your code base to these systems in the free versions that are then going to further use that for training data that may produce output for other people, that may be something you want to be weary of, and definitely worry about what PII information you might be providing.

If you want to avoid that altogether, there are some tools like LM Studio, Twinny, Cody.dev. These can run LMs-- so not LLMs. So an LLM is the large language model. An LM is just a language model. It's a pared-down version, but they can run locally. I've run them on my Mac laptop. This is ideal for development. And it can plug into IDEs with-- LM Studio allows you to plug into VS Code or JetBrains, and that all happens locally.

Now, it's going to take up some of your computer memory, so it may be a little slow, but you don't have to worry about sending data to anybody. It's all happening locally using the versions of the models you have. So the output might not be as sophisticated, but then it's secure because it's not going anywhere. It's just in your system, and then you can review it and see what it's outputting.

So some takeaways here-- generative AI can free up time for high-level tasks. The goal of your teams is, one, to know how to use your tools. But really, to be able to use those tools to let the school's mission and values be shown to the world, be shown to the right audiences. So using generative AI tools can allow you to do more of that and focus on doing that and using AI to augment the work that has to be done for your programming, for your web development, to accomplish those goals.

You want to discuss as a team which AI tools are a best fit for your needs. Every team is made up of different skill sets, and levels, and roles. So think, as a team, what do you already have covered? What roles are you good on with having humans do it? But what places maybe you're not fully covered. Do you not have a UX? Do you not have a UI person? You may want to use AI tools to help augment that.

So talk through as a team where you find gaps, where you think you're weakest on, where you want to add some improvements to, and look for what AI tools can help support that.

Always be cautious of and verify the output of any generative AI tool you use. Again, think of these as artificial interns. So whatever you tell them to do there, they're going to do it. How well they do it-- that's up to how well you told them what to do. But check their work-- just don't blindly believe it and then send it out to the world. You want to verify that it does what it needs to do, it does it the way you expect it to, and that you've had some sort of insight into it before you release it.

I think this piece here kind of goes against the new methodology of vibe coding, which I've tried for some things. If you haven't heard about that, that's just where you just describe to the AI code tool what you want the functionality to do, and you don't even look at the code. You just let it generate it, and you see, yes, that looks correct. I'm going to go with that. I find a lot of problems with that, so I always try to verify the output.

Then you want to confirm the rules and regulations for your institute. What are you allowed to use AI for, and what are you not? In my talks with people in higher ed, a lot of the institution AI guidelines right now are really focused on faculty and students for good reason. That's the majority of people that we are concerned about how they're using AI.

But you may want to talk about-- and if you're in the IT group, come up with guidelines on how you can use AI for other purposes, like web development. So at MIT Sloan, I've had my team talk with our central IT group about-- how are we able to use AI to generate code? What do we have to be concerned about? What level of data are we allowed to use AI with? Not sensitive information, but low-level public information-- can we use it for that?

And so having that understanding of what the rules and regulations are, or even still developing the rules and regulations, are very important, and you want to keep that in mind.

That's what I have. Again, these slides are available at bit.ly/UMassUnlockAI. And I'll come back to this slide in a moment, but I just want to share-- when you view this, I have a link to all the tools I've shared or talked about, as well as some further reading for research and information about using AI for programming.

And then, if you're curious, all the images in this presentation I generated using AI through a service called NightCafe Studio. And I have all the prompts available so you can see how I prompted the AI to generate these images.

And with that, I believe we have some time for questions. And thank you.

Thank you, Mike. I'm sure you're probably updating this pretty frequently given the changing landscape.

Yes. [CHUCKLES]

We have a few questions. First one, I've heard that it's not advisable to input university codes, data, content, questions, and similar material into open-source AI web tools for review due to potential absorption into the models and associated security concerns. Could you provide some insights on this issue and how other universities have addressed it?

Yeah, that's a great question, and definitely right on. You want to be careful of what systems you're using, what services you're using, and what data you're providing to it. Some of the way universities are resolving this-- I can talk a little bit about what MIT is doing-- but you may set up agreements with the companies that are providing these.

For example, MIT, we use Microsoft products, so I have access to Microsoft Copilot. And I know the university has set up rules in their agreements with Microsoft to say, anything we provide you, you are not allowed to use for further training. And I think most of the services, if you're not using the free versions, like if you're submitting to an API-- if you're using a paid version, there are options for turning off, you consume my data and use it for further training.

So definitely read the terms of service and check with those. Again, if you're using a local tool like Twinny or Cody.dev that runs locally, you don't have to worry about that because it's not sending it anywhere. So another option is, can you run it locally? Or if you have to use a third-party service, what agreements can the university put in place to make sure data is not being consumed for further training?

Excellent. Hello again, everyone from this morning. I'm jumping in again. Ron's computer locked up a little bit, so we're going through and seeing what we have. We have some posted here, Mike. When should we consider using generative AI instead of machine learning and other tools? Leadership teams are pushing AI when non-AI tools can do the job, but it's all about the buzzwords.

Yeah, yeah. [CHUCKLES] There's a lot of buzzwords, right? AI is in everything now. It's like everything's gluten-free, everything is powered by AI. So when to use machine learning versus generative AI? I don't think I'm the expert to answer that sort of question. I would say that's part of the duty, I would say, of us as developers of understanding the landscape of the tools available to us and being able to explain, yeah, we can use AI for this. Or no, that doesn't really make sense for these reasons.

And being confident in providing those answers because you're the expert in this field, where someone else in the university who are making these requests, they may not be the expert. So learning how to explain the differences in further research of when something might be useful or not, I guess, is the best advice I can give.

All right, that's great. And another one-- since models only produce output that looks correct, should we expect AI to answer students' questions correctly? For example, if a student asks an AI chatbot on the UMass website what the tuition for a specific program is, will it be correct every time? Is AI suitable for this type of task?

Yeah, I think AI is suitable for that sort of task. And about it being correct every time-- this is the interesting thing. If you want to implement, say, an AI chatbot on the UMass website so that students can ask these questions, how do you test that? Because every time you ask it, it's going to produce slightly different output. So it depends on the tool you're using and the models in which it produces output.

So, for example, I would say in this use case, you want a AI tool that uses a RAG model, which means you can point it to knowledge bases-- say, like, the school's website-- and say, only deliver your answers from this content. And then it will use the general models that it has to format those answers so that if someone asks you-- you won't say this specifically-- but when somebody asks, what programs do you offer? It's going to use the data it's fed in from, say, the website content to produce those answers.

But definitely training and testing a lot just to make sure it's generally correct is the most important thing. A good example of this-- if you look up some videos about the YaleSights team-- the team at Yale who built their own internal chatbot for students to use-- they have some presentations out there on how they went about training the data, training the tool, and testing it.

So definitely testing and checking on how you can lock in where it's getting its sources. You don't want to use a tool that just uses all the internet data because then it might hallucinate what you offer. So if you can limit where it's getting its knowledge base from, that's the most helpful thing.

Awesome. That's great. We're at time. I want to be fair-- I went from the bottom up for this-- so let me just ask the second part of this last question here, Mike. A short second's answer here.

Yeah.

Academic institutions-- are there any concerns about LMs, agentic AI being trained with plagiarized work when higher ed typically takes a dim view of the practice?

100%. Yep. And there's a lot of articles out there about-- especially finding students are using AI to generate potentially plagiarized work, but then using AI to ask it not to be plagiarized. And I don't know how we solve that. There's lots of tools out there. But that's definitely a big concern that's currently being looked into by a lot of universities.

Awesome Thank you, Mike. I am a huge nerd, so I enjoyed the hell out of this presentation. I'm sure others did as well. Thank you so much for this. I am absolutely going to go find those slides now and get some of those links. The local LMs are going to be great for that. So this is wonderful. Thank you so much.

Thank you again for having me.

Awesome.