[Protocol technical on-boarding series] Rust tips and tricks (March 3rd, 2023)

Captions as YouTube publishes them, usually machine-generated, so product and people names can be misheard.

1. 0:03

   cool so um I'm going to present about uh well this is a fairly introductionary presentation on uh R some tips and tricks that may come useful to you working uh at Pagoda on near core or any of the adjacent libraries uh crates um this is roughly just a collection of what I thought might be useful to you uh ultimately there are so many things to keep in mind that uh definitely not all of them will fit um so at the end I'm going to put some link

2. 0:42

   to places where we keep our exhaustive uh guidance and suggestions um but still anyhow it's kind of probably useful to have this um presentation just describing you what the reasoning behind those ideas are so just to kick things off the ground let's talk about correctness we care about correctness a lot at near at pagod uh if things go wrong for whatever reason that's going to be bad for the

3. 1:16

   chain uh we may have some sort of a vulnerability uh that may cause an aage or uh somebody's token getting stolen and uh we all know how that goes early uh they are not there have been plenty of projects so far that have suffered uh in one way or another and had their token value just collapsing or just the entire project going poop uh because somebody something wrong went so ensuring

4. 1:47

   correctness of our code is one of those things that we must to uh keep in mind every uh all the time as we develop near core so the way uh I arithmetic let's talk about arithmetics then the way arithmetics are approached are very similar between all kinds of languages and rest is not uh an exception here uh we do have a operator like a plus or minus uh we put that in between two variables A and B uh and outome the sum

5. 2:20

   or the difference or the multiplication result uh that you can then assign to a variable uh and use it for for the compu python does have some additional uh operators that are not displayed here so like for example exponation uh which is represented by double star uh and in Converse programming languages as haskill has showen to not assign a a symbol to operations such such as

6. 2:52

   remainder so you see an empty window here uh but by far and large uh similarities are abundant um but well I wouldn't be really talking about metics and operators if they were really that straightforward right uh so let's take a look at an example in this example I uh Define a test case uh with a uh hashtag uh and then I don't know which braces these are the pointy braces uh and between the point way we have test uh that makes a function uh a test function that gets

7. 3:34

   one when you run a cargo test uh and within this I'm going to uh Define some specification for the function that I'm going to implement a little bit later and the intent is to have a uh function which gets a element from the end of the list uh given the index so uh given the list that is one two three at index Z there's going to be three so the last element at index two we're going to have one which is the first element and uh

8. 4:07

   higher indexes are out of bound so we're going to be returning none and to implement this uh I kind of intentionally made a mistake here in a comparison operator uh just to demonstrate the point uh but ultimately uh I feel like this code is something that anybody could have written uh and made this mistake generally without realizing it uh where you just use the wrong comparison operator instead of more an equal than uh length we used

9. 4:40

   more than length uh and we will see uh and then we later use uh these variables in a computation which does uh which subtracts from the length of the array the index and minus one because uh this is zero indexed so if you have a length of three and index is two then you need to to get a index of zero you still need to subtract the one um and before we go into that and aside the couple of ways you can run this example uh one way is you can create a

10. 5:20

    grate put this code into liberas and invoke cargo test uh this will work a lot of the time uh especially works well when you have uh a dependency on an external quate uh it also uh works better if you have a complicated Test example uh that you want to try out but for this case for example when our test case is just a single function or two with a test function as well it can be easier to just compile it to

11. 5:53

    trust C directly uh this work great if you don't need any other great on the standard Library such as the case right here uh or alternatively we can use the rest playground which is a uh website where you can prototype and play with code uh some of the potential benefits of using that are the fact that you can share links to the code uh examples with to anybody else uh and I will be doing this uh the same throughout this presentation so as you are looking uh to

12. 6:29

    the notes and to the presentation feel free to click on any of these links to look at the code directly uh run the code experiment with it change it see how it behaves and uh yeah that Prett much the EXP do definitely explore the functionality that this Tool uh makes available to you because it's is fairly uh featureful so going back into the arithmetic operations this uh fails well as I explained a little bit previously I kind of intentionally fud

13. 7:07

    with the comparison operator there uh so what happened was that uh now we are subtracting the uh total length of the O uh such that it eventually ends up overflowing so in this case uh we are subtracting from Zer a minus one and what happens is uh well it goes below zero and un signed integers cannot represent that very well here I kind of highlighted the areas where the mistake has

14. 7:45

    occurred uh I could pick this code by changing the comparison operator to be the correct one uh so greater than or equal but I think there is definitely a better approach there that uh was to forge you and is not really available in other languages or as common at least uh and this is to use methods that the standard Library provide to apply various checked operation so it does provide more than just checking it can ultimately the list of operator that

15. 8:25

    are a made available to you are addition subtraction multiplication the various uh typical arithmetic operation that you might expect and each of them have a uh multiple different handling modes so in this case we are using check uh which verifies that uh the operation will not overflow and if it does overflow it returns a none otherwise it returns a sum that you can extract the computation value out of it uh the question mark

16. 8:59

    marked in Orange uh then unwrap this result if it was none it return from the function if it was some it just gives you the uh result of subraction the difference back um some of the handling mode of handling handling are thing called saturation so for example if subtracting from a number would result in a number less than zero for sign numbers uh in the end the result would become zero overall you won't have to deal with uh none or some

17. 9:37

    wrapping but it has some different uh semantics right so you need to think about which one uh which mode of handling is uh most appropriate for you marked in blue I also used a different method called get uh instead of indexing a vector or a slide directly uh this methods similarly is a uh checking method in that instead of panicking when uh the index is out of bound it uh Returns the some or none it

18. 10:11

    wasn't strictly appropriate in this example I would say because uh in this case if the index is out of bound of the slide that we are trying to index uh it's a programming error a logic error in this function and so uh arguably panicking might have been a better idea really uh mostly because we have already checked for the indexing uh in disas uh beforehand with check sub function let's talk about a related uh operation which is

19. 10:52

    casts uh they I feel uh share a fair amount of similar Falls as a uh typical arithmetic but at least they do not panic when you overflow or do something wrong with them uh they typically default to uh handling the problem by wrapping around uh and if you are not expecting this to happen this can be a this can cause a logicor and zone so yeah intuitively you would want to use the very basic that is made available in R which is value as b b

20. 11:33

    being the type that you want to convert to um but I do think there's a better approach here um but before that so here's a list of or a table that I prepared uh that demonstrates which casts are lossless so they do not suffer from the problem that I just described so so you always can C to the type that is itself so if a number is you8 an unsigned 8 bit integer then casting it to an unsigned 8bit integer is always going to succeed and

21. 12:13

    so it will succeed if you cast it to Aion and integer such as say U6 or U 128 or U size um on the other hand if you cast you A2 and signed 8 bit integer now it's going to be a little bit more tricky because uh if you have a uh value such as a 255 which is the maximum that unsigned 8bit integer can hold uh a signed 8bit integer cannot hold it so you get going to get a wrap around and

22. 12:44

    the actual value that will be stored is going to be minus one which is not probably not what you might have wanted uh so in order to deal with that uh much like with uh the method for arithmetics the standard Library provides some TW uh such as from and into uh for conversion that cannot fail or between between type that cannot fail so if you have a u8 uh you can be sure that uh if you

23. 13:18

    call do into onto on that value uh the conversion will always be successful and lless to the whatever type you are converting uh this way uh similarly for type that uh do not Implement from an into and uh do the conventions are not lot they also try from and try into methods uh those do the checking for the potential overflow within themselves so you don't really need to worry about implementing that

24. 13:50

    correctly uh by open coding it uh much like I demonstrated with arithmetic uh operation that can be become a problem when you for example mistakenly use a a wrong kind of comparison operator so what we have uh looked at so far if you find yourself open coding checks on metic operations uh cast your code may be better uh off if it if you replaced uh those operations with a method like check sub or a conversion uh

25. 14:26

    TW such as from try from in to try into um similarly if you find yourself open coding check sound indexing uh you may be better Serve by a method such as say slid colon colon get and in general I would say uh typically this uh kind of pattern appli to many different things so like if you find yourself open coding something that might very well be a method that already does this for you and Implement to check internally so you don't really need to worry about getting the details

26. 15:02

    right it doesn't always make 100% sense to apply those methods indiscriminantly so as I have uh explained with the uh indexing with get for example uh using a proper indexing uh that panics may have been better there because uh it would have CAU any potential implementation uh mistake within the function that we were implementing and use R playground to prototype share Snippets uh play with code experiment with it uh it's a fairly easy and convenient way to do it and

27. 15:41

    finally uh test attribute for lightweight unit test are very uh common and L code uh do definitely read on that because we use that throughout our near core code base very extensively as well let's talk about Expressions loops and structure goto so unlike many other languages that you may have encountered before us uh what is a very expression oriented language and by expression I mean that there exps are things that produce a

28. 16:25

    result or you may be used to a uh terms such as an R value right value uh value that appear appears on the right side of an assignment operator uh so in these other languages you things like uh arithmetics so you have uh a plus b that's an expression uh in wor many more things are expressions as well so in this example you see here uh we are writing this code in a fairly non-expression wise way so wolf and

29. 17:04

    silence are an expression uh that is assigned to the result uh the result is a variable uh that is being assigned to and we eventually print it but there is a better way to write this with being an expression oriented language if is also an expression so both bodies of it whichever executes are the last expression within it is also a result of the if expression as a v so instead of worrying whether the result has been

30. 17:37

    assigned in uh you might have forgotten to assign to it in one of the benches right uh if you write the code this way you won't forget because ultimately the then will check oh hey if I am in the uh to Branch then I am expecting this to return a string but also I'm expecting this to produce a string in the El Branch as well so um in case you forget the compiler will remind you and it's all I would say pretty much always

31. 18:08

    recommended to uh see if you can write your code this way rather than the other way um just so because it's a little bit more Rusty uh a little bit more uh check going to be applied to your code uh and this semantic analysis that was does execute on your code will have an easier time proving that your code is in fact correct um and to extend on this idea Loop is also an expression and by loop I

32. 18:46

    I mean here the loop a loop statement expression and R WR it and like Loops in general such as a v or for Loops um so if you ever find yourself writing some sort of a search algorithm or something along those lines and have a result result variable out of line outside of this Loop that you would eventually assign the result to um you may have a easier time if you replace this with a low Lev perimeter which is just the basic Loop uh and that because when you break

33. 19:26

    out of it you have an opport Unity to provide the value uh that will then become a result of the loop expression as a row so in this example we are breaking out of the loop with break 42 and eventually the result also become 42 because that how loops work and rest um do further on that idea even more um you can actually Implement a structure control flow with this and use that as an as a replacement

34. 20:08

    for goto statement that are available in C or C++ and although those specific statements are very heavily discouraged because they are very difficult to reason about um that's not really the case and was because due to the structured nature you cannot really jump to another arbitary location from another arbitrary location it enforces some specific rule that make uh such a control flow much easier to reason

35. 20:39

    about um so in this example we have a while loop that is uh checking a done variable um which is a search so whenever we find a value we will assign the done uh to true and then eventually this file Loop will also stop uh the problem with this piece of code is that every iteration we are checking on a variable uh and if we had an access to go to if it was not heavily discouraged or something then well technically we could jump out of this

36. 21:15

    Loop immediately after we are we have found our value that would potentially make the code more efficient although U optimizers are probably pretty good at optimizing this particular pattern uh but well demonstration purposes call for uh simplification because well slide do not really have much space for them for extensive pieces of code so we will have to work with this so in Rust you can instead use uh labels

37. 21:54

    and uh break again to basically terminate the loop in any place within that Loop so that would make this control uh this control flow contract structure so you cannot really break into this loop from some other location by calling breakout or from somewhere else in your code it's only going to work within a loop that you're already breaking out of and if you look at this example in particular

38. 22:27

    you will noted that this is still a nested Loop so uh we have this outer loop that we have labeled but then uh we also within the execute a for Loop and you will notice that this break out the statement will break out of these two Loops together so if you ever find yourself in a situation where you do want to have a behavior like that uh consider just adding a label to your uh loop or a block blocks also work fine you can do that

39. 22:58

    with a V Loop or for loop as well just flap a uh Prime outer or prime whatever name you want just like a lifetime colum before your UH semi statement the loop or while loop or full loop or block or anything like that uh and you will have an opportunity then to uh break out of it uh wherever you are with it so uh next on the list of topics is a nam spacing this came up in a fairly recent discussion on zulip that we had uh

40. 23:37

    where people were asking how should I oppose name facing and I felt like this is a very good topic to cover in this presentation as well so where I'm coming from readability I feel is is always more important than uh ease of wri writing the code for the first time and that because well you mostly write the code once maybe you modify it twice and then every time you look at it again it's going to be reading it so optim the way you should optimize for this is for the reading

41. 24:16

    experience rather than from from the writing or modification perspective um and with that in mind using partially qualified name uh I feel like they can really help with readability and understandability of the code uh mostly just because you don't really need to go back to the top of the file to look at like okay what has been imported into this file and what this variable or name will the first

42. 24:47

    to so instead you can just write Apple colon colon p or rly column column Pi sorry about that and people will be able to tell that oh hey this is coming from the apple crate or this one this other use is coming from the r quate or module or whatever and you will not need to have this contextual knowledge of where this code is or what the file has already imported to actually understand what

43. 25:17

    this code really does or means um implementing display is a perfect example that I come across uh specifically in this local reasoning from this local reasoning perspective so I might uh import STD colon colon fmt format uh at the top of the file but think about what would happen if I uh actually imported all of the things within it that I used so display result formatter uh maybe form matter is one of those things that are not really all that ambiguous uh

44. 25:58

    it's formatter so it's probably going to come from the module intended Library called format right uh display is also fairly uh common name that you might want to import but think about the result there are uh multiple or use the name result in rest is fairly common this result might refer to many different things it might refer to local areas that you have the find it might refer to uh the typical thing that you import from the standard library to its Prelude by not importing anything which

45. 26:37

    has this uh the two parameter for the okay value or the error value or will it might refer to this uh result alas imported from the formatting Library so I would argue that having just fmt call and call and result and the return position here is just making the code significantly more understandable without much loss and like ability or speed you write the code really like it's five additional

46. 27:11

    characters it's not really all that hard right so definitely consider thinking about like just how deep or how deep do you want to import right what point does it make to um actually if it would make potentially to qualify this code and the way I personally approach this is I write the code largely qualified to the to the point where I think makes sense all the time and then when I find Opportunities to unqualified so to import names uh a little bit more specifically at the top

47. 27:49

    of the file then I will use uh the ID uh rest analyzer to actually replace the uh use of the name with the import and the shorten name instead on the other in the other direction you can use uh use to simplify the match pattern so for example uh as we have in here we do repeat the word by a huge number of times right um it's not really giving much value to us if we knew that this is a pie for the first match uh for

48. 28:30

    the first pattern we probably will know that it is a pie for the next three right we will remember as we read the code since it is fairly local to the to this code itself right so it's not like going to go to the next area of the file by itself and so the thing that you can do is actually just above the match import for the all of the variants of the P enum in this case uh that will let you simplify your match patterns a little bit uh you will be able to say okay Apple raspberry uh

49. 29:10

    blueberry without prefixing that necessarily with the word spy that is not giving much uh additional information to you and even then if you do this um the import of the variant is fairly local to the match statement right because it's just above it so it's probably not going to get lost or be confusing in that kind of context so the second checkpoint uh the code is written once or maybe a couple times and uh then

50. 29:48

    later it's going to be read many times over so uh definitely optimize for the later uh it definit defitely makes most sense at least to me and I think it probably does make sense to uh apply this kind of reasoning in general uh R does have a concept of structure go to uh and an ability to set results to Expressions uh that are not expression in other languages so uh that may enable you to simplify code or write

51. 30:28

    it in a ways uh that are much more readable than in other languages um I find that using Loop for example is giving me like a light bulb above my head in terms of oh hey I can do that and it make the code like just a pleasure to read as well similarly on the readability topic R name space as R name spaces I need uh but it is important just to consider the context and to try a couple different things out to see the best way to apply

52. 31:08

    them uh importing all the things to the their like most deep level is not necessarily the best approach all the time sometimes applying some name pacing uh partial qualification is uh a good idea and make the go easier to understand sometimes it is uh that you want to actually qualify or import thing that you wouldn't necessarily import like variant of an Em Right so definitely consider uh thinking about

53. 31:41

    that as you write your code and now going on to uh topics that are a little bit more advanced um I generally just picked one uh thing that came to my mind kind of like out of like the blue sky um I didn't really uh consider other options uh well there are really really many different topic that I could have talked about this seemed like a uh easy enough to GA one um and yet it is still fairly

54. 32:19

    obscure that I would technically call it semi Advanced trust and it is about um use of impul trade you can use this pattern of impul and trade here is a name of the trade of any trade so it could have been display as we have already covered it could be something like iterator as is uh seen in this example and here we are using this pattern in a return position so a function to return is a imple iterator uh and items of these iterators

55. 32:55

    are strings here we Implement a f algorithm or a game or a coding challenge I don't know back in a couple of years ago I heard this being a fairly popular uh interview challenge um not so much right now uh but still it serves as a very nice uh example and presentation and so the problem here is that we want to return and iterate that returns as fiz buzz buzz Fizz or the number that is uh if it's nther three of those um and typically well you could

56. 33:39

    try to write an actual name type which would be something like map uh within it there would be um the range that we are mapping over but unfortunately in this specific case we cannot do that because uh the map the the map type includes the type of this closure which is uh the conention the Transformer of the numbers uh in the range and in you cannot really name the type of the closure it's if you see if you ever see an error meage you will see something like in the uh pointy backet

57. 34:23

    there's going to be uh closure something something uh there's going to be probably an address of the closure something along those lines uh ultimately uh this is not a valid rust syntax because well there's no way to name a closure type and rust and so you do have to sort to letting the compiler to figure out what the type is for you uh and that's exactly what the impul iterator here does in this uh specific return position uh but it also like generally

58. 35:01

    uh really really helps if you just need to uh I just don't want to bother writing down a very long time that you could write down but you just can't wrap your head around it would be very verbos or just like really hard to figure out um imput grate in that case is very nice um Sim Illy you can put Temple TR in an argument position such as I have done here uh this one is a little bit less of

59. 35:37

    a magic it's just a Syntax suggar for uh introducing a generic so I could have U added iterator as a typical generic to this function sorry about that uh and it will work just pretty much the same uh that generic would have been bounded over the iterator that return the thing this does pretty much the same and uh now in the function body we can iterate over the values within that iterator and print them out um so not much of a

60. 36:16

    surprise here it's just nice knowing that you can do this because it is a little bit more than uh just introducing a generic proper um and then finally like I would say using a generic in that function does not necessarily the best idea and that because uh in a function called print fbz we probably want to be printing F buuz rather than an arbitrary values of an iterator that iterates over the

61. 36:51

    strings right um but in adding a generic this is not necessarily what happens it allows you to pass in any iterator that uh produces any strings and it would print out arbitary stuff for you um so unfortunately this is a unstable feature so we are going to uh backck open a Pandora box a Pandora chest um for a preview of what uh what in the future might look like and you do that by using the nightly

62. 37:31

    compiler and adding this uh feature attribute to equate in this case we are using type alas imput dat uh which is standing for type alas equals imple trade um and what this feature does is that it will pick a single type such as has been done uh when we were using inut rate in the done position such as here and we are doing the same here here with fbz uh as a return value of the fbz function it will figure out the exact

63. 38:13

    type of this uh that this function is ret turning and then you can refer to that exact type again u in the argument position of the print FB uh function now it will no longer allow you to pass in the impul any other impul iterator than what has been returned from this fbus function again this is a fairly I would say feature that you probably won't be needing much especially because it is unstable right now but it is still

64. 38:50

    something useful uh keeping in mind as a concern if you are designing an API or something along those lines like really think about well if I'm taking this as an argument does it really make sense for my library what will happen if somebody will pass something else than what I expect um and these kind of concerns are alleviated by uh imple trade sometime other times you may want

65. 39:14

    to introduce a uh separate type of your own maybe uh that might be even a better option than using uh type area simple tra and the first plate but well I went to show off uh unstable R so this kind of ended up being the uh example I went with so conclusions out of these uh uh out of this section on semi Advan rust is that rust is still faing you and and a language that evolves fairly quickly

66. 39:50

    still um you may encounter paino that are potentially about to be solved by a feature that is not yet available to uh the table compiler um it does not prevent you from playing with those features eventually you uh there not one and not two projects out there that just use the nightly compiler for these features because uh they make your code more expressive more powerful you can express safety and variant that are not

67. 40:20

    otherwise possible there are types you cannot name in West and features there are some there are some features that deal with that problem right impul trate is one of those um you can potentially uh try to get out of this situation by using uh generics in a fairly uh ingenious way so for example uh you can it is possible to return a closure from a function without being able to

68. 40:52

    name it by introducing a generic but it's really not all that easy to figure out really um and might involve things or trick like having a call back uh closure in there somewhere so do keep an eye out for new features uh to make your code more powerful and you might notice that a fair number of features and what are just a synx sugar that is the case what I demonstrated with imple uh trade in the argument position but also a little

69. 41:31

    bit less intuitively this also holds true for the four or V Loop which do eventually dgar to like the regular Loop uh admittedly the compiler does uh introduce some additional semantic checking over that but uh it is a fairly straightforward conversion really just repl the while loop Loop and condition with a regular Loop and then regular if within there that check the same condition and breaks out otherwise if it not

70. 42:07

    passing and now for the last stretch of this presentation on some quick miscellaneous tips do definitely check out uh the third party cargo command maybe you will find something to augment your workflow uh make you more productive uh there are plenty of those on qu. um this link lead to a uh search that you can look at maybe you will find something that I have in discover for example uh do definitely share if you

71. 42:40

    find something uh interesting because the ecosystem is always evolving and tools are getting better and better and well my toolbox may have gotten out of dat and some amount of time as well um to access documentation uh in the browser uh search field you can just write doc. theate name it will open you a page uh from do project which is a part of project maintained by the web language uh

72. 43:19

    Community um for the standard Library the create name is STD for the core Library the create name is core um some quit that you may commonly see in use and near core are surday uh clap that kind of stuff replace the quate with that quate name and you will immediately end up with a documentation for that great most most recent version to uh do familiarize yourself with the St Style Guidelines and as well as the rest of the books uh within which these

73. 43:54

    guidelines are uh there is a huge amount of very useful information within that book that we uh keep up to dat all the time so this presentation will eventually potentially end up a little bit taale uh the documentation within that book will stay out stay up to date all the time especially with regard to uh specifics of development uh on neore and surrounding quid and last but not

74. 44:28

    least if you do have any question hit some snag uh encounter some issue or just a St uh package your code example uh in a wor playground and drop drop into like the hashtag r on zulip uh and we will definitely be there to help you out uh to look at your code suggest and a best approach for your problem uh we are definitely open to talking about rust in general like if you have some uh concerns or you like rust or you found something interesting uh this stream is

75. 45:07

    definitely the one to post into uh to share your experiences and that kind of stuff as well and that's all from me unfortunately so much for presenting to me but now we have it recorded for everybody that was really great thank you so much uh cool thank you for watching